Surgical methods for control of one visualization with another

ABSTRACT

In general, devices, systems, and methods for control of one visualization with another are provided.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Prov. Pat. App. No.63/249,658 entitled “Surgical Devices, Systems, And Methods For ControlOf One Visualization With Another” filed Sep. 29, 2021, which is herebyincorporated by reference in its entirety.

FIELD

The present disclosure relates generally to surgical devices, systems,and methods for control of one visualization with another.

BACKGROUND

Surgical systems often incorporate an imaging system, which can allowmedical practitioners to view a surgical site and/or one or moreportions thereof on one or more displays, e.g., a monitor, a computertablet screen, etc. The display(s) can be local and/or remote to asurgical theater. The imaging system can include a scope with a camerathat views the surgical site and transmits the view to the one or moredisplays viewable by medical practitioner(s).

Imaging systems can be limited by the information that they are able torecognize and/or convey to the medical practitioner(s). For example,certain concealed structures, physical contours, and/or dimensionswithin a three-dimensional space may be unrecognizable intraoperativelyby certain imaging systems. For another example, certain imaging systemsmay be incapable of communicating and/or conveying certain informationto the medical practitioner(s) intraoperatively.

Accordingly, there remains a need for improved surgical imaging.

SUMMARY

In general, devices, systems, and methods for control of onevisualization with another are provided.

In another embodiment, a surgical method includes deliveringradiofrequency (RF) energy to tissue at a surgical site with a firstelectrode array of a surgical device engaging the tissue between jaws ofthe surgical device, monitoring a parameter of non-targeted tissue atthe surgical site using a second electrode array, and adjusting, with acontroller, the energy delivery to the tissue based on the monitoredparameter.

The method can vary in any number of ways. For example, the secondelectrode array can include a filter or a gating element that preventsthe energy delivered by the first electrode array from infiltrating thesecond electrode array. For another example, a return path of the firstelectrode array can be separate from a return path of the secondelectrode array. For yet another example, the second electrode array canmonitors at least one of impedance, frequency response, capacitance,temperature, and pressure of the non-targeted tissue, and the controllercan be configured to adjust the energy delivery based on the monitoredat least one of impedance, frequency response, capacitance, temperature,and pressure. For another example, the adjusting can include adjusting avariable parameter of a control algorithm of the surgical device, andthe method can further include executing the control algorithm includingthe adjusted variable parameter, thereby affecting the energy deliveryfrom the first electrode array to the tissue. For yet another example, asurgical hub can include the controller. For still another example, arobotic surgical system can include the controller, and the surgicaldevice is releasably coupled to and controlled by the robotic surgicalsystem.

In another aspect, a surgical system is provided that in one embodimentincludes a surgical device including first and second jaws configured toengage a target tissue therebetween. The target tissue is at a surgicalsite, and the surgical device includes a first electrode arrayconfigured to deliver radiofrequency (RF) energy to the target tissue.The system also includes a second electrode array configured to monitor,during the energy delivery, a non-targeted tissue at the surgical site,and a controller configured to control the energy delivery of the firstelectrode array based on the monitoring of the second tissue by thesecond electrode array.

The system can have any number of variations. For example, the secondelectrode array can include a filter or a gating element configured toprevent the energy delivered by the first electrode array frominfiltrating the second electrode array. For another example, a returnpath of the first electrode array can be separate from a return path ofthe second electrode array. For yet another example, the secondelectrode array can includes a temperature sensor configured to monitora temperature of the non-targeted tissue, and the controller can beconfigured to adjust the energy delivery in response to the monitoredtemperature being greater than a predetermined threshold temperature.For still another example, the control can include the controllercontrolling power level and frequency of the energy delivery. Foranother example, the control can include the controller controllingfrequency of the energy delivery. For yet another example, the secondelectrode array can be configured to monitor impedance of thenon-targeted tissue, and the controller can be configured to adjust theenergy delivery in response to the monitored impedance as compared to athreshold impedance. For another example, the second electrode array canbe configured to monitor a frequency response of the non-targetedtissue, and the controller can be configured to adjust the energydelivery based on the frequency response. For still another example, thesecond electrode array can be configured to monitor at least one ofcapacitance and pressure of the non-targeted tissue, and the controllercan be configured to adjust the energy delivery based on the monitoredat least one of capacitance and pressure. For another example, thecontroller can be configured to cause the control by adjusting avariable parameter of a control algorithm of the surgical device, andthe control algorithm can be configured to, when executed, affect theenergy delivery from the first electrode array to the tissue. For yetanother example, a surgical hub can include the controller. For stillanother example, a robotic surgical system can include the controller,and the surgical device can be configured to releasably couple to and becontrolled by the robotic surgical system.

BRIEF DESCRIPTION OF DRAWINGS

The present invention is described by way of reference to theaccompanying figures which are as follows:

FIG. 1 is a schematic view of one embodiment of a surgical visualizationsystem;

FIG. 2 is a schematic view of triangularization between a surgicaldevice, an imaging device, and a critical structure of FIG. 1 ;

FIG. 3 is a schematic view of another embodiment of a surgicalvisualization system;

FIG. 4 is a schematic view of one embodiment of a control system for asurgical visualization system;

FIG. 5 is a schematic view of one embodiment of a control circuit of acontrol system for a surgical visualization system;

FIG. 6 is a schematic view of one embodiment of a combinational logiccircuit of a surgical visualization system;

FIG. 7 is a schematic view of one embodiment of a sequential logiccircuit of a surgical visualization system;

FIG. 8 is a schematic view of yet another embodiment of a surgicalvisualization system;

FIG. 9 is a schematic view of another embodiment of a control system fora surgical visualization system;

FIG. 10 is a graph showing wavelength versus absorption coefficient forvarious biological materials;

FIG. 11 is a schematic view of one embodiment of a spectral emittervisualizing a surgical site;

FIG. 12 is a graph depicting illustrative hyperspectral identifyingsignatures to differentiate a ureter from obscurants;

FIG. 13 is a graph depicting illustrative hyperspectral identifyingsignatures to differentiate an artery from obscurants;

FIG. 14 is a graph depicting illustrative hyperspectral identifyingsignatures to differentiate a nerve from obscurants;

FIG. 15 is a schematic view of one embodiment of a near infrared (NIR)time-of-flight measurement system being utilized intraoperatively;

FIG. 16 shows a time-of-flight timing diagram for the system of FIG. 15;

FIG. 17 is a schematic view of another embodiment of a near infrared(NIR) time-of-flight measurement system being utilized intraoperatively;

FIG. 18 is a schematic view of one embodiment of a computer-implementedinteractive surgical system;

FIG. 19 is a schematic view of one embodiment a surgical system beingused to perform a surgical procedure in an operating room;

FIG. 20 is a schematic view of one embodiment of a surgical systemincluding a smart surgical instrument and a surgical hub;

FIG. 21 is a flowchart showing a method of controlling the smartsurgical instrument of FIG. 20 ;

FIG. 22 is a schematic view of a colon illustrating major resections ofthe colon;

FIG. 23 is a perspective partial cross-sectional view of one embodimentof a duodenal mucosal resurfacing procedure;

FIG. 24 is a perspective view of one embodiment of an adjunct;

FIG. 25 is a perspective view of a portion of the adjunct of FIG. 24 ;

FIG. 26 is a perspective view of another embodiment of an adjunct;

FIG. 27 is a perspective view of yet another embodiment of an adjunct;

FIG. 28 illustrates one embodiment of controlling energy of an ablationdevice;

FIG. 29 is a schematic cross-sectional view of one embodiment of aflexible force probe pressing on a tissue wall in which a scope ispositioned;

FIG. 30 is a perspective view of the probe and the scope of FIG. 29 ;

FIG. 31 is a schematic cross-sectional view of a body lumen having firstand second electrodes positioned therein;

FIG. 32 is a view of the first electrode and a portion of the body lumenof FIG. 31 over four time points;

FIG. 33 is a graph showing temperature, estimated thickness, power, andimpedance versus time including the four time points of FIG. 32 ;

FIG. 34 is a perspective view of a distal portion of one embodiment ofan ablation device in a compressed configuration;

FIG. 35 is a perspective view of the distal portion the ablation deviceof FIG. 34 in an expanded configuration;

FIG. 36 is a perspective view of the ablation device of FIG. 35positioned relative to a tumor;

FIG. 37 is a perspective view of a distal portion of another embodimentof an ablation device;

FIG. 38 is a perspective partially cross-sectional view of oneembodiment of a scope and ablation device positioned in a body lumen andan imaging device positioned external to the body lumen;

FIG. 39 is a schematic cross-sectional view of a portion of FIG. 38 ;

FIG. 40 is a graph showing time versus power, tissue impedance, tissuetemperature, and electrode pressure with respect to the embodiment ofFIG. 38 ;

FIG. 41 is another schematic cross-sectional view of a body lumen havingfirst and second electrodes positioned therein;

FIG. 42 is a graph showing temperature and power with respect to theembodiment of FIG. 41 ;

FIG. 43 is yet another schematic cross-sectional view of a body lumenhaving first and second electrodes positioned therein;

FIG. 44 is a graph showing temperature and power with respect to theembodiment of FIG. 43 ;

FIG. 45 is a perspective partially cross-sectional view of oneembodiment of a scope and ablation device positioned in a body lumen andfiber optic sensors positioned external to the body lumen;

FIG. 46 is a graph showing temperature and power with respect to theembodiment of FIG. 45 ;

FIG. 47 is a schematic view of one embodiment of an ablation device in afirst state of expansion and with four electrodes of the ablation devicedelivering energy;

FIG. 48 is a schematic view of the ablation device of FIG. 47 in asecond state of expansion and with the four electrodes deliveringenergy;

FIG. 49 is a schematic view of the ablation device of FIG. 47 in thefirst state of expansion and with two of the four electrodes deliveringenergy;

FIG. 50 is a schematic view of the ablation device of FIG. 47 in thesecond state of expansion and with three of the four electrodesdelivering energy;

FIG. 51 is a side schematic view of one embodiment of an end effectorincluding upper and lower jaws;

FIG. 52 is a side schematic view of another embodiment of an endeffector including upper and lower jaws in an open position with tissuepositioned between the upper and lower jaws;

FIG. 53 is a cross-sectional view of one embodiment of an electrodeconfiguration of the end effector of FIG. 52 with the upper and lowerjaws in a closed position;

FIG. 53A is a schematic view of the electrode configuration and tissueof FIG. 53 ;

FIG. 54 is another cross-sectional view of the upper and lower jaws andthe tissue of FIG. 53 ;

FIG. 55 is a cross-sectional view of another embodiment of an electrodeconfiguration of the end effector of FIG. 52 with the upper and lowerjaws in a closed position;

FIG. 56 is a another cross-sectional view of the upper and lower jawsand the tissue of FIG. 55 ;

FIG. 57 is a cross-sectional view of yet another embodiment of anelectrode configuration of the end effector of FIG. 52 with the upperand lower jaws in a closed position;

FIG. 58 is another cross-sectional view of the upper and lower jaws andthe tissue of FIG. 57 ;

FIG. 59 is one embodiment of a series of pulses that can be applied totissue;

FIG. 60 is a flowchart of one embodiment of a process of using frequencyresponse to monitor tissue between therapeutic energy applications;

FIG. 61 is a schematic diagram showing one embodiment of multi-frequencyapplication in the process of FIG. 60 ;

FIG. 62 is a schematic diagram of the process of FIG. 60 ;

FIG. 63 illustrates one embodiment of a variable frequency measurementpulse and a therapeutic treatment pulse that can be applied to tissue;

FIG. 64 is a schematic diagram illustrating one embodiment of a processof using the pulses of FIG. 63 ;

FIG. 65 illustrates one embodiment of a high frequency measurement pulseand a therapeutic treatment pulse that can be applied to tissue;

FIG. 66 illustrates a measured acceptable condition of the highfrequency measurement pulse of FIG. 65 ;

FIG. 67 illustrates a measured fault condition of the high frequencymeasurement pulse of FIG. 65 ;

FIG. 68 illustrates a measured marginal condition of the high frequencymeasurement pulse of FIG. 65 ;

FIG. 69 illustrates one embodiment of first and second measurementpulses and a therapeutic treatment pulse that can be applied to tissue;

FIG. 70 is a schematic diagram of one embodiment of providing a variablefrequency measurement pulse;

FIG. 71 is another schematic diagram of a portion of the diagram of FIG.70 ;

FIG. 72 is a perspective view of a distal portion of another embodimentof an ablation device;

FIG. 73 is a perspective partial cross-sectional view of one embodimentof an ablation device including a first magnet positioned inside aduodenum and a surgical device including a second magnet positionedoutside the duodenum;

FIG. 74 is a cross-sectional view of the first magnet of FIG. 73 in apassive configuration in the duodenum;

FIG. 75 is a cross-sectional view of the first magnet of FIG. 73 in anattraction configuration in the duodenum;

FIG. 76 is a cross-sectional view of the first magnet of FIG. 73 in arepulsion configuration in the duodenum;

FIG. 77 is a side schematic partial cross-sectional view of anotherembodiment of an ablation device extending distally from a scope locatedwithin a hollow organ or body lumen;

FIG. 78 is a perspective partial transparent view of the scope and theablation device of FIG. 77 with a magnet positioned around an externalsurface of the hollow organ or body lumen;

FIG. 79 is a side schematic partial cross-sectional view of the scope,the ablation device, and the magnet moved between first and secondpositions;

FIG. 80 is a graph showing tissue thickness and magnetic attraction forthree positions of the scope of FIG. 77 ; and

FIG. 81 is a side partial cross-sectional view of an ultrasound imagingdevice visualizing through a tissue wall.

DETAILED DESCRIPTION

Certain exemplary embodiments will now be described to provide anoverall understanding of the principles of the structure, function,manufacture, and use of the devices, systems, and methods disclosedherein. One or more examples of these embodiments are illustrated in theaccompanying drawings. A person skilled in the art will understand thatthe devices, systems, and methods specifically described herein andillustrated in the accompanying drawings are non-limiting exemplaryembodiments and that the scope of the present invention is definedsolely by the claims. The features illustrated or described inconnection with one exemplary embodiment may be combined with thefeatures of other embodiments. Such modifications and variations areintended to be included within the scope of the present invention.

Further, in the present disclosure, like-named components of theembodiments generally have similar features, and thus within aparticular embodiment each feature of each like-named component is notnecessarily fully elaborated upon. Additionally, to the extent thatlinear or circular dimensions are used in the description of thedisclosed systems, devices, and methods, such dimensions are notintended to limit the types of shapes that can be used in conjunctionwith such systems, devices, and methods. A person skilled in the artwill recognize that an equivalent to such linear and circular dimensionscan easily be determined for any geometric shape. A person skilled inthe art will appreciate that a dimension may not be a precise value butnevertheless be considered to be at about that value due to any numberof factors such as manufacturing tolerances and sensitivity ofmeasurement equipment. Sizes and shapes of the systems and devices, andthe components thereof, can depend at least on the size and shape ofcomponents with which the systems and devices will be used.

Surgical Visualization

In general, a surgical visualization system is configured to leverage“digital surgery” to obtain additional information about a patient'sanatomy and/or a surgical procedure. The surgical visualization systemis further configured to convey data to one or more medicalpractitioners in a helpful manner Various aspects of the presentdisclosure provide improved visualization of the patient's anatomyand/or the surgical procedure, and/or use visualization to provideimproved control of a surgical tool (also referred to herein as a“surgical device” or a “surgical instrument”).

“Digital surgery” can embrace robotic systems, advanced imaging,advanced instrumentation, artificial intelligence, machine learning,data analytics for performance tracking and benchmarking, connectivityboth inside and outside of the operating room (OR), and more. Althoughvarious surgical visualization systems described herein can be used incombination with a robotic surgical system, surgical visualizationsystems are not limited to use with a robotic surgical system. Incertain instances, surgical visualization using a surgical visualizationsystem can occur without robotics and/or with limited and/or optionalrobotic assistance. Similarly, digital surgery can occur withoutrobotics and/or with limited and/or optional robotic assistance.

In certain instances, a surgical system that incorporates a surgicalvisualization system may enable smart dissection in order to identifyand avoid critical structures. Critical structures include anatomicalstructures such as a ureter, an artery such as a superior mesentericartery, a vein such as a portal vein, a nerve such as a phrenic nerve,and/or a tumor, among other anatomical structures. In other instances, acritical structure can be a foreign structure in the anatomical field,such as a surgical device, a surgical fastener, a clip, a tack, abougie, a band, a plate, and other foreign structures. Criticalstructures can be determined on a patient-by-patient and/or aprocedure-by-procedure basis. Smart dissection technology may provide,for example, improved intraoperative guidance for dissection and/or mayenable smarter decisions with critical anatomy detection and avoidancetechnology.

A surgical system incorporating a surgical visualization system mayenable smart anastomosis technologies that provide more consistentanastomoses at optimal location(s) with improved workflow. Cancerlocalization technologies may be improved with a surgical visualizationplatform. For example, cancer localization technologies can identify andtrack a cancer location, orientation, and its margins. In certaininstances, the cancer localization technologies may compensate formovement of a surgical instrument, a patient, and/or the patient'sanatomy during a surgical procedure in order to provide guidance back tothe point of interest for medical practitioner(s).

A surgical visualization system may provide improved tissuecharacterization and/or lymph node diagnostics and mapping. For example,tissue characterization technologies may characterize tissue type andhealth without the need for physical haptics, especially when dissectingand/or placing stapling devices within the tissue. Certain tissuecharacterization technologies may be utilized without ionizing radiationand/or contrast agents. With respect to lymph node diagnostics andmapping, a surgical visualization platform may, for example,preoperatively locate, map, and ideally diagnose the lymph system and/orlymph nodes involved in cancerous diagnosis and staging.

During a surgical procedure, information available to a medicalpractitioner via the “naked eye” and/or an imaging system may provide anincomplete view of the surgical site. For example, certain structures,such as structures embedded or buried within an organ, can be at leastpartially concealed or hidden from view. Additionally, certaindimensions and/or relative distances can be difficult to ascertain withexisting sensor systems and/or difficult for the “naked eye” toperceive. Moreover, certain structures can move pre-operatively (e.g.,before a surgical procedure but after a preoperative scan) and/orintraoperatively. In such instances, the medical practitioner can beunable to accurately determine the location of a critical structureintraoperatively.

When the position of a critical structure is uncertain and/or when theproximity between the critical structure and a surgical tool is unknown,a medical practitioner's decision-making process can be inhibited. Forexample, a medical practitioner may avoid certain areas in order toavoid inadvertent dissection of a critical structure; however, theavoided area may be unnecessarily large and/or at least partiallymisplaced. Due to uncertainty and/or overly/excessive exercises incaution, the medical practitioner may not access certain desiredregions. For example, excess caution may cause a medical practitioner toleave a portion of a tumor and/or other undesirable tissue in an effortto avoid a critical structure even if the critical structure is not inthe particular area and/or would not be negatively impacted by themedical practitioner working in that particular area. In certaininstances, surgical results can be improved with increased knowledgeand/or certainty, which can allow a surgeon to be more accurate and, incertain instances, less conservative/more aggressive with respect toparticular anatomical areas.

A surgical visualization system can allow for intraoperativeidentification and avoidance of critical structures. The surgicalvisualization system may thus enable enhanced intraoperative decisionmaking and improved surgical outcomes. The surgical visualization systemcan provide advanced visualization capabilities beyond what a medicalpractitioner sees with the “naked eye” and/or beyond what an imagingsystem can recognize and/or convey to the medical practitioner. Thesurgical visualization system can augment and enhance what a medicalpractitioner is able to know prior to tissue treatment (e.g.,dissection, etc.) and, thus, may improve outcomes in various instances.As a result, the medical practitioner can confidently maintain momentumthroughout the surgical procedure knowing that the surgicalvisualization system is tracking a critical structure, which may beapproached during dissection, for example. The surgical visualizationsystem can provide an indication to the medical practitioner insufficient time for the medical practitioner to pause and/or slow downthe surgical procedure and evaluate the proximity to the criticalstructure to prevent inadvertent damage thereto. The surgicalvisualization system can provide an ideal, optimized, and/orcustomizable amount of information to the medical practitioner to allowthe medical practitioner to move confidently and/or quickly throughtissue while avoiding inadvertent damage to healthy tissue and/orcritical structure(s) and, thus, to minimize the risk of harm resultingfrom the surgical procedure.

Surgical visualization systems are described in detail below. Ingeneral, a surgical visualization system can include a first lightemitter configured to emit a plurality of spectral waves, a second lightemitter configured to emit a light pattern, and a receiver, or sensor,configured to detect visible light, molecular responses to the spectralwaves (spectral imaging), and/or the light pattern. The surgicalvisualization system can also include an imaging system and a controlcircuit in signal communication with the receiver and the imagingsystem. Based on output from the receiver, the control circuit candetermine a geometric surface map, e.g., three-dimensional surfacetopography, of the visible surfaces at the surgical site and a distancewith respect to the surgical site, such as a distance to an at leastpartially concealed structure. The imaging system can convey thegeometric surface map and the distance to a medical practitioner. Insuch instances, an augmented view of the surgical site provided to themedical practitioner can provide a representation of the concealedstructure within the relevant context of the surgical site. For example,the imaging system can virtually augment the concealed structure on thegeometric surface map of the concealing and/or obstructing tissuesimilar to a line drawn on the ground to indicate a utility line belowthe surface. Additionally or alternatively, the imaging system canconvey the proximity of a surgical tool to the visible and obstructingtissue and/or to the at least partially concealed structure and/or adepth of the concealed structure below the visible surface of theobstructing tissue. For example, the visualization system can determinea distance with respect to the augmented line on the surface of thevisible tissue and convey the distance to the imaging system.

Throughout the present disclosure, any reference to “light,” unlessspecifically in reference to visible light, can include electromagneticradiation (EMR) or photons in the visible and/or non-visible portions ofthe EMR wavelength spectrum. The visible spectrum, sometimes referred toas the optical spectrum or luminous spectrum, is that portion of theelectromagnetic spectrum that is visible to (e.g., can be detected by)the human eye and may be referred to as “visible light” or simply“light.” A typical human eye will respond to wavelengths in air that arefrom about 380 nm to about 750 nm. The invisible spectrum (e.g., thenon-luminous spectrum) is that portion of the electromagnetic spectrumthat lies below and above the visible spectrum. The invisible spectrumis not detectable by the human eye. Wavelengths greater than about 750nm are longer than the red visible spectrum, and they become invisibleinfrared (IR), microwave, and radio electromagnetic radiation.Wavelengths less than about 380 nm are shorter than the violet spectrum,and they become invisible ultraviolet, x-ray, and gamma rayelectromagnetic radiation.

FIG. 1 illustrates one embodiment of a surgical visualization system100. The surgical visualization system 100 is configured to create avisual representation of a critical structure 101 within an anatomicalfield. The critical structure 101 can include a single criticalstructure or a plurality of critical structures. As discussed herein,the critical structure 101 can be any of a variety of structures, suchas an anatomical structure, e.g., a ureter, an artery such as a superiormesenteric artery, a vein such as a portal vein, a nerve such as aphrenic nerve, a vessel, a tumor, or other anatomical structure, or aforeign structure, e.g., a surgical device, a surgical fastener, asurgical clip, a surgical tack, a bougie, a surgical band, a surgicalplate, or other foreign structure. As discussed herein, the criticalstructure 101 can be identified on a patient-by-patient and/or aprocedure-by-procedure basis. Embodiments of critical structures and ofidentifying critical structures using a visualization system are furtherdescribed in U.S. Pat. No. 10,792,034 entitled “Visualization OfSurgical Devices” issued Oct. 6, 2020, which is hereby incorporated byreference in its entirety.

In some instances, the critical structure 101 can be embedded in tissue103. The tissue 103 can be any of a variety of tissues, such as fat,connective tissue, adhesions, and/or organs. Stated differently, thecritical structure 101 may be positioned below a surface 105 of thetissue 103. In such instances, the tissue 103 conceals the criticalstructure 101 from the medical practitioner's “naked eye” view. Thetissue 103 also obscures the critical structure 101 from the view of animaging device 120 of the surgical visualization system 100. Instead ofbeing fully obscured, the critical structure 101 can be partiallyobscured from the view of the medical practitioner and/or the imagingdevice 120.

The surgical visualization system 100 can be used for clinical analysisand/or medical intervention. In certain instances, the surgicalvisualization system 100 can be used intraoperatively to providereal-time information to the medical practitioner during a surgicalprocedure, such as real-time information regarding proximity data,dimensions, and/or distances. A person skilled in the art willappreciate that information may not be precisely real time butnevertheless be considered to be real time for any of a variety ofreasons, such as time delay induced by data transmission, time delayinduced by data processing, and/or sensitivity of measurement equipment.The surgical visualization system 100 is configured for intraoperativeidentification of critical structure(s) and/or to facilitate theavoidance of the critical structure(s) 101 by a surgical device. Forexample, by identifying the critical structure 101, a medicalpractitioner can avoid maneuvering a surgical device around the criticalstructure 101 and/or a region in a predefined proximity of the criticalstructure 101 during a surgical procedure. For another example, byidentifying the critical structure 101, a medical practitioner can avoiddissection of and/or near the critical structure 101, thereby helping toprevent damage to the critical structure 101 and/or helping to prevent asurgical device being used by the medical practitioner from beingdamaged by the critical structure 101.

The surgical visualization system 100 is configured to incorporatetissue identification and geometric surface mapping in combination withthe surgical visualization system's distance sensor system 104. Incombination, these features of the surgical visualization system 100 candetermine a position of a critical structure 101 within the anatomicalfield and/or the proximity of a surgical device 102 to the surface 105of visible tissue 103 and/or to the critical structure 101. Moreover,the surgical visualization system 100 includes an imaging system thatincludes the imaging device 120 configured to provide real-time views ofthe surgical site. The imaging device 120 can include, for example, aspectral camera (e.g., a hyperspectral camera, multispectral camera, orselective spectral camera), which is configured to detect reflectedspectral waveforms and generate a spectral cube of images based on themolecular response to the different wavelengths. Views from the imagingdevice 120 can be provided in real time to a medical practitioner, suchas on a display (e.g., a monitor, a computer tablet screen, etc.). Thedisplayed views can be augmented with additional information based onthe tissue identification, landscape mapping, and the distance sensorsystem 104. In such instances, the surgical visualization system 100includes a plurality of subsystems—an imaging subsystem, a surfacemapping subsystem, a tissue identification subsystem, and/or a distancedetermining subsystem. These subsystems can cooperate tointra-operatively provide advanced data synthesis and integratedinformation to the medical practitioner.

The imaging device 120 can be configured to detect visible light,spectral light waves (visible or invisible), and a structured lightpattern (visible or invisible). Examples of the imaging device 120includes scopes, e.g., an endoscope, an arthroscope, an angioscope, abronchoscope, a choledochoscope, a colonoscope, a cytoscope, aduodenoscope, an enteroscope, an esophagogastro-duodenoscope(gastroscope), a laryngoscope, a nasopharyngo-neproscope, asigmoidoscope, a thoracoscope, an ureteroscope, or an exoscope. Scopescan be particularly useful in minimally invasive surgical procedures. Inopen surgery applications, the imaging device 120 may not include ascope.

The tissue identification subsystem can be achieved with a spectralimaging system. The spectral imaging system can rely on imaging such ashyperspectral imaging, multispectral imaging, or selective spectralimaging. Embodiments of hyperspectral imaging of tissue are furtherdescribed in U.S. Pat. No. 9,274,047 entitled “System And Method ForGross Anatomic Pathology Using Hyperspectral Imaging” issued Mar. 1,2016, which is hereby incorporated by reference in its entirety.

The surface mapping subsystem can be achieved with a light patternsystem. Various surface mapping techniques using a light pattern (orstructured light) for surface mapping can be utilized in the surgicalvisualization systems described herein. Structured light is the processof projecting a known pattern (often a grid or horizontal bars) on to asurface. In certain instances, invisible (or imperceptible) structuredlight can be utilized, in which the structured light is used withoutinterfering with other computer vision tasks for which the projectedpattern may be confusing. For example, infrared light or extremely fastframe rates of visible light that alternate between two exact oppositepatterns can be utilized to prevent interference. Embodiments of surfacemapping and a surgical system including a light source and a projectorfor projecting a light pattern are further described in U.S. Pat. Pub.No. 2017/0055819 entitled “Set Comprising A Surgical Instrument”published Mar. 2, 2017, U.S. Pat. Pub. No. 2017/0251900 entitled“Depiction System” published Sep. 7, 2017, and U.S. Pat. Pub. No.2021/0196385 entitled “Surgical Systems For Generating Three DimensionalConstructs Of Anatomical Organs And Coupling Identified AnatomicalStructures Thereto” published Jul. 1, 2021, which are herebyincorporated by reference in their entireties.

The distance determining system can be incorporated into the surfacemapping system. For example, structured light can be utilized togenerate a three-dimensional (3D) virtual model of the visible surface105 and determine various distances with respect to the visible surface105. Additionally or alternatively, the distance determining system canrely on time-of-flight measurements to determine one or more distancesto the identified tissue (or other structures) at the surgical site.

The surgical visualization system 100 also includes a surgical device102. The surgical device 102 can be any suitable surgical device.Examples of the surgical device 102 includes a surgical dissector, asurgical stapler, a surgical grasper, a clip applier, a smoke evacuator,a surgical energy device (e.g., mono-polar probes, bi-polar probes,ablation probes, an ultrasound device, an ultrasonic end effector,etc.), etc. In some embodiments, the surgical device 102 includes an endeffector having opposing jaws that extend from a distal end of a shaftof the surgical device 102 and that are configured to engage tissuetherebetween.

The surgical visualization system 100 can be configured to identify thecritical structure 101 and a proximity of the surgical device 102 to thecritical structure 101. The imaging device 120 of the surgicalvisualization system 100 is configured to detect light at variouswavelengths, such as visible light, spectral light waves (visible orinvisible), and a structured light pattern (visible or invisible). Theimaging device 120 can include a plurality of lenses, sensors, and/orreceivers for detecting the different signals. For example, the imagingdevice 120 can be a hyperspectral, multispectral, or selective spectralcamera, as described herein. The imaging device 120 can include awaveform sensor 122 (such as a spectral image sensor, detector, and/orthree-dimensional camera lens). For example, the imaging device 120 caninclude a right-side lens and a left-side lens used together to recordtwo two-dimensional images at the same time and, thus, generate a 3Dimage of the surgical site, render a three-dimensional image of thesurgical site, and/or determine one or more distances at the surgicalsite. Additionally or alternatively, the imaging device 120 can beconfigured to receive images indicative of the topography of the visibletissue and the identification and position of hidden criticalstructures, as further described herein. For example, a field of view ofthe imaging device 120 can overlap with a pattern of light (structuredlight) on the surface 105 of the tissue 103, as shown in FIG. 1 .

As in this illustrated embodiment, the surgical visualization system 100can be incorporated into a robotic surgical system 110. The roboticsurgical system 110 can have a variety of configurations, as discussedherein. In this illustrated embodiment, the robotic surgical system 110includes a first robotic arm 112 and a second robotic arm 114. Therobotic arms 112, 114 each include rigid structural members 116 andjoints 118, which can include servomotor controls. The first robotic arm112 is configured to maneuver the surgical device 102, and the secondrobotic arm 114 is configured to maneuver the imaging device 120. Arobotic control unit of the robotic surgical system 110 is configured toissue control motions to the first and second robotic arms 112, 114,which can affect the surgical device 102 and the imaging device 120,respectively.

In some embodiments, one or more of the robotic arms 112, 114 can beseparate from the main robotic system 110 used in the surgicalprocedure. For example, at least one of the robotic arms 112, 114 can bepositioned and registered to a particular coordinate system without aservomotor control. For example, a closed-loop control system and/or aplurality of sensors for the robotic arms 112, 114 can control and/orregister the position of the robotic arm(s) 112, 114 relative to theparticular coordinate system. Similarly, the position of the surgicaldevice 102 and the imaging device 120 can be registered relative to aparticular coordinate system.

Examples of robotic surgical systems include the Ottava™robotic-assisted surgery system (Johnson & Johnson of New Brunswick,N.J.), da Vinci® surgical systems (Intuitive Surgical, Inc. ofSunnyvale, Calif.), the Hugo™ robotic-assisted surgery system (MedtronicPLC of Minneapolis, Minn.), the Versius® surgical robotic system (CMRSurgical Ltd of Cambridge, UK), and the Monarch® platform (Auris Health,Inc. of Redwood City, Calif.). Embodiments of various robotic surgicalsystems and using robotic surgical systems are further described in U.S.Pat. Pub. No. 2018/0177556 entitled “Flexible Instrument Insertion UsingAn Adaptive Force Threshold” filed Dec. 28, 2016, U.S. Pat. Pub. No.2020/0000530 entitled “Systems And Techniques For Providing MultiplePerspectives During Medical Procedures” filed Apr. 16, 2019, U.S. Pat.Pub. No. 2020/0170720 entitled “Image-Based Branch Detection And MappingFor Navigation” filed Feb. 7, 2020, U.S. Pat. Pub. No. 2020/0188043entitled “Surgical Robotics System” filed Dec. 9, 2019, U.S. Pat. Pub.No. 2020/0085516 entitled “Systems And Methods For Concomitant MedicalProcedures” filed Sep. 3, 2019, U.S. Pat. No. 8,831,782 entitled“Patient-Side Surgeon Interface For A Teleoperated Surgical Instrument”filed Jul. 15, 2013, and Intl. Pat. Pub. No. WO 2014151621 entitled“Hyperdexterous Surgical System” filed Mar. 13, 2014, which are herebyincorporated by reference in their entireties.

The surgical visualization system 100 also includes an emitter 106. Theemitter 106 is configured to emit a pattern of light, such as stripes,grid lines, and/or dots, to enable the determination of the topographyor landscape of the surface 105. For example, projected light arrays 130can be used for three-dimensional scanning and registration on thesurface 105. The projected light arrays 130 can be emitted from theemitter 106 located on the surgical device 102 and/or one of the roboticarms 112, 114 and/or the imaging device 120. In one aspect, theprojected light array 130 is employed by the surgical visualizationsystem 100 to determine the shape defined by the surface 105 of thetissue 103 and/or motion of the surface 105 intraoperatively. Theimaging device 120 is configured to detect the projected light arrays130 reflected from the surface 105 to determine the topography of thesurface 105 and various distances with respect to the surface 105.

As in this illustrated embodiment, the imaging device 120 can include anoptical waveform emitter 123, such as by being mounted on or otherwiseattached on the imaging device 120. The optical waveform emitter 123 isconfigured to emit electromagnetic radiation 124 (near-infrared (NIR)photons) that can penetrate the surface 105 of the tissue 103 and reachthe critical structure 101. The imaging device 120 and the opticalwaveform emitter 123 can be positionable by the robotic arm 114. Theoptical waveform emitter 123 is mounted on or otherwise on the imagingdevice 120 but in other embodiments can be positioned on a separatesurgical device from the imaging device 120. A corresponding waveformsensor 122 (e.g., an image sensor, spectrometer, or vibrational sensor)of the imaging device 120 is configured to detect the effect of theelectromagnetic radiation received by the waveform sensor 122. Thewavelengths of the electromagnetic radiation 124 emitted by the opticalwaveform emitter 123 are configured to enable the identification of thetype of anatomical and/or physical structure, such as the criticalstructure 101. The identification of the critical structure 101 can beaccomplished through spectral analysis, photo-acoustics, and/orultrasound, for example. In one aspect, the wavelengths of theelectromagnetic radiation 124 can be variable. The waveform sensor 122and optical waveform emitter 123 can be inclusive of a multispectralimaging system and/or a selective spectral imaging system, for example.In other instances, the waveform sensor 122 and optical waveform emitter123 can be inclusive of a photoacoustic imaging system, for example.

The distance sensor system 104 of the surgical visualization system 100is configured to determine one or more distances at the surgical site.The distance sensor system 104 can be a time-of-flight distance sensorsystem that includes an emitter, such as the emitter 106 as in thisillustrated embodiment, and that includes a receiver 108. In otherinstances, the time-of-flight emitter can be separate from thestructured light emitter. The emitter 106 can include a very tiny lasersource, and the receiver 108 can include a matching sensor. The distancesensor system 104 is configured to detect the “time of flight,” or howlong the laser light emitted by the emitter 106 has taken to bounce backto the sensor portion of the receiver 108. Use of a very narrow lightsource in the emitter 106 enables the distance sensor system 104 todetermining the distance to the surface 105 of the tissue 103 directlyin front of the distance sensor system 104.

The receiver 108 of the distance sensor system 104 is positioned on thesurgical device 102 in this illustrated embodiment, but in otherembodiments the receiver 108 can be mounted on a separate surgicaldevice instead of the surgical device 102. For example, the receiver 108can be mounted on a cannula or trocar through which the surgical device102 extends to reach the surgical site. In still other embodiments, thereceiver 108 for the distance sensor system 104 can be mounted on aseparate robotically-controlled arm of the robotic system 110 (e.g., onthe second robotic arm 114) than the first robotic arm 112 to which thesurgical device 102 is coupled, can be mounted on a movable arm that isoperated by another robot, or be mounted to an operating room (OR) tableor fixture. In some embodiments, the imaging device 120 includes thereceiver 108 to allow for determining the distance from the emitter 106to the surface 105 of the tissue 103 using a line between the emitter106 on the surgical device 102 and the imaging device 120. For example,a distance d_(e) can be triangulated based on known positions of theemitter 106 (on the surgical device 102) and the receiver 108 (on theimaging device 120) of the distance sensor system 104. The 3D positionof the receiver 108 can be known and/or registered to the robotcoordinate plane intraoperatively.

As in this illustrated embodiment, the position of the emitter 106 ofthe distance sensor system 104 can be controlled by the first roboticarm 112, and the position of the receiver 108 of the distance sensorsystem 104 can be controlled by the second robotic arm 114. In otherembodiments, the surgical visualization system 100 can be utilized apartfrom a robotic system. In such instances, the distance sensor system 104can be independent of the robotic system.

In FIG. 1 , distance d_(e) is emitter-to-tissue distance from theemitter 106 to the surface 105 of the tissue 103, and distance d_(t) isdevice-to-tissue distance from a distal end of the surgical device 102to the surface 105 of the tissue 103. The distance sensor system 104 isconfigured to determine the emitter-to-tissue distance d_(e). Thedevice-to-tissue distance d_(t) is obtainable from the known position ofthe emitter 106 on the surgical device 102, e.g., on a shaft thereofproximal to the surgical device's distal end, relative to the distal endof the surgical device 102. In other words, when the distance betweenthe emitter 106 and the distal end of the surgical device 102 is known,the device-to-tissue distance d_(t) can be determined from theemitter-to-tissue distance d_(e). In some embodiments, the shaft of thesurgical device 102 can include one or more articulation joints and canbe articulatable with respect to the emitter 106 and jaws at the distalend of the surgical device 102. The articulation configuration caninclude a multi-joint vertebrae-like structure, for example. In someembodiments, a 3D camera can be utilized to triangulate one or moredistances to the surface 105.

In FIG. 1 , distance d_(w) is camera-to-critical structure distance fromthe optical waveform emitter 123 located on the imaging device 120 tothe surface of the critical structure 101, and distance d_(A) is a depthof the critical structure 101 below the surface 105 of the tissue 103(e.g., the distance between the portion of the surface 105 closest tothe surgical device 102 and the critical structure 101). Thetime-of-flight of the optical waveforms emitted from the opticalwaveform emitter 123 located on the imaging device 120 are configured todetermine the camera-to-critical structure distance d_(w).

As shown in FIG. 2 , the depth d_(A) of the critical structure 101relative to the surface 105 of the tissue 103 can be determined bytriangulating from the camera-to-critical structure distance d_(w) andknown positions of the emitter 106 on the surgical device 102 and theoptical waveform emitter 123 on the imaging device 120 (and, thus, theknown distance d_(x) therebetween) to determine distance d_(y), which isthe sum of the distances d_(e) and d_(A). Additionally or alternatively,time-of-flight from the optical waveform emitter 123 can be configuredto determine the distance from the optical waveform emitter 123 to thesurface 105 of the tissue 103. For example, a first waveform (or rangeof waveforms) can be utilized to determine the camera-to-criticalstructure distance d_(w) and a second waveform (or range of waveforms)can be utilized to determine the distance to the surface 105 of thetissue 103. In such instances, the different waveforms can be utilizedto determine the depth of the critical structure 101 below the surface105 of the tissue 103.

Additionally or alternatively, the distance d_(A) can be determined froman ultrasound, a registered magnetic resonance imaging (MRI), orcomputerized tomography (CT) scan. In still other instances, thedistance d_(A) can be determined with spectral imaging because thedetection signal received by the imaging device 120 can vary based onthe type of material, e.g., type of the tissue 103. For example, fat candecrease the detection signal in a first way, or a first amount, andcollagen can decrease the detection signal in a different, second way,or a second amount.

In another embodiment of a surgical visualization system 160 illustratedin FIG. 3 , a surgical device 162, and not the imaging device 120,includes the optical waveform emitter 123 and the waveform sensor 122that is configured to detect the reflected waveforms. The opticalwaveform emitter 123 is configured to emit waveforms for determining thedistances d_(t) and d_(w) from a common device, such as the surgicaldevice 162, as described herein. In such instances, the distance d_(A)from the surface 105 of the tissue 103 to the surface of the criticalstructure 101 can be determined as follows:

d _(A) =d _(w) −d _(t)

The surgical visualization system 100 includes a control systemconfigured to control various aspects of the surgical visualizationsystem 100. FIG. 4 illustrates one embodiment of a control system 133that can be utilized as the control system of the surgical visualizationsystem 100 (or other surgical visualization system described herein).The control system 133 includes a control circuit 132 configured to bein signal communication with a memory 134. The memory 134 is configuredto store instructions executable by the control circuit 132, such asinstructions to determine and/or recognize critical structures (e.g.,the critical structure 101 of FIG. 1 ), instructions to determine and/orcompute one or more distances and/or three-dimensional digitalrepresentations, and instructions to communicate information to amedical practitioner. As in this illustrated embodiment, the memory 134can store surface mapping logic 136, imaging logic 138, tissueidentification logic 140, and distance determining logic 141, althoughthe memory 134 can store any combinations of the logics 136, 138, 140,141 and/or can combine various logics together. The control system 133also includes an imaging system 142 including a camera 144 (e.g., theimaging system including the imaging device 120 of FIG. 1 ), a display146 (e.g., a monitor, a computer tablet screen, etc.), and controls 148of the camera 144 and the display 146. The camera 144 includes an imagesensor 135 (e.g., the waveform sensor 122) configured to receive signalsfrom various light sources emitting light at various visible andinvisible spectra (e.g., visible light, spectral imagers,three-dimensional lens, etc.). The display 146 is configured to depictreal, virtual, and/or virtually-augmented images and/or information to amedical practitioner.

In an exemplary embodiment, the image sensor 135 is a solid-stateelectronic device containing up to millions of discrete photodetectorsites called pixels. The image sensor 135 technology falls into one oftwo categories: Charge-Coupled Device (CCD) and Complementary MetalOxide Semiconductor (CMOS) imagers and more recently, short-waveinfrared (SWIR) is an emerging technology in imaging. Another type ofthe image sensor 135 employs a hybrid CCD/CMOS architecture (sold underthe name “sCMOS”) and consists of CMOS readout integrated circuits(ROICs) that are bump bonded to a CCD imaging substrate. CCD and CMOSimage sensors are sensitive to wavelengths in a range of about 350 nm toabout 1050 nm, such as in a range of about 400 nm to about 1000 nm. Aperson skilled in the art will appreciate that a value may not beprecisely at a value but nevertheless considered to be about that valuefor any of a variety of reasons, such as sensitivity of measurementequipment and manufacturing tolerances. CMOS sensors are, in general,more sensitive to IR wavelengths than CCD sensors. Solid state imagesensors are based on the photoelectric effect and, as a result, cannotdistinguish between colors. Accordingly, there are two types of colorCCD cameras: single chip and three-chip. Single chip color CCD camerasoffer a common, low-cost imaging solution and use a mosaic (e.g., Bayer)optical filter to separate incoming light into a series of colors andemploy an interpolation algorithm to resolve full color images. Eachcolor is, then, directed to a different set of pixels. Three-chip colorCCD cameras provide higher resolution by employing a prism to directeach section of the incident spectrum to a different chip. More accuratecolor reproduction is possible, as each point in space of the object hasseparate RGB intensity values, rather than using an algorithm todetermine the color. Three-chip cameras offer extremely highresolutions.

The control system 133 also includes an emitter (e.g., the emitter 106)including a spectral light source 150 and a structured light source 152each operably coupled to the control circuit 133. A single source can bepulsed to emit wavelengths of light in the spectral light source 150range and wavelengths of light in the structured light source 152 range.Alternatively, a single light source can be pulsed to provide light inthe invisible spectrum (e.g., infrared spectral light) and wavelengthsof light on the visible spectrum. The spectral light source 150 can be,for example, a hyperspectral light source, a multispectral light source,and/or a selective spectral light source. The tissue identificationlogic 140 is configured to identify critical structure(s) (e.g., thecritical structure 101 of FIG. 1 ) via data from the spectral lightsource 150 received by the image sensor 135 of the camera 144. Thesurface mapping logic 136 is configured to determine the surfacecontours of the visible tissue (e.g., the tissue 103) based on reflectedstructured light. With time-of-flight measurements, the distancedetermining logic 141 is configured to determine one or more distance(s)to the visible tissue and/or the critical structure. Output from each ofthe surface mapping logic 136, the tissue identification logic 140, andthe distance determining logic 141 is configured to be provided to theimaging logic 138, and combined, blended, and/or overlaid by the imaginglogic 138 to be conveyed to a medical practitioner via the display 146of the imaging system 142.

The control circuit 132 can have a variety of configurations. FIG. 5illustrates one embodiment of a control circuit 170 that can be used asthe control circuit 132 configured to control aspects of the surgicalvisualization system 100. The control circuit 170 is configured toimplement various processes described herein. The control circuit 170includes a microcontroller that includes a processor 172 (e.g., amicroprocessor or microcontroller) operably coupled to a memory 174. Thememory 174 is configured to store machine-executable instructions that,when executed by the processor 172, cause the processor 172 to executemachine instructions to implement various processes described herein.The processor 172 can be any one of a number of single-core or multicoreprocessors known in the art. The memory 174 can include volatile andnon-volatile storage media. The processor 172 includes an instructionprocessing unit 176 and an arithmetic unit 178. The instructionprocessing unit 176 is configured to receive instructions from thememory 174.

The surface mapping logic 136, the imaging logic 138, the tissueidentification logic 140, and the distance determining logic 141 canhave a variety of configurations. FIG. 6 illustrates one embodiment of acombinational logic circuit 180 configured to control aspects of thesurgical visualization system 100 using logic such as one or more of thesurface mapping logic 136, the imaging logic 138, the tissueidentification logic 140, and the distance determining logic 141. Thecombinational logic circuit 180 includes a finite state machine thatincludes a combinational logic 182 configured to receive data associatedwith a surgical device (e.g. the surgical device 102 and/or the imagingdevice 120) at an input 184, process the data by the combinational logic182, and provide an output 184 to a control circuit (e.g., the controlcircuit 132).

FIG. 7 illustrates one embodiment of a sequential logic circuit 190configured to control aspects of the surgical visualization system 100using logic such as one or more of the surface mapping logic 136, theimaging logic 138, the tissue identification logic 140, and the distancedetermining logic 141. The sequential logic circuit 190 includes afinite state machine that includes a combinational logic 192, a memory194, and a clock 196. The memory 194 is configured to store a currentstate of the finite state machine. The sequential logic circuit 190 canbe synchronous or asynchronous. The combinational logic 192 isconfigured to receive data associated with a surgical device (e.g. thesurgical device 102 and/or the imaging device 120) at an input 426,process the data by the combinational logic 192, and provide an output499 to a control circuit (e.g., the control circuit 132). In someembodiments, the sequential logic circuit 190 can include a combinationof a processor (e.g., processor 172 of FIG. 5 ) and a finite statemachine to implement various processes herein. In some embodiments, thefinite state machine can include a combination of a combinational logiccircuit (e.g., the combinational logic circuit 192 of FIG. 7 ) and thesequential logic circuit 190.

FIG. 8 illustrates another embodiment of a surgical visualization system200. The surgical visualization system 200 is generally configured andused similar to the surgical visualization system 100 of FIG. 1 , e.g.,includes a surgical device 202 and an imaging device 220. The imagingdevice 220 includes a spectral light emitter 223 configured to emitspectral light in a plurality of wavelengths to obtain a spectral imageof hidden structures, for example. The imaging device 220 can alsoinclude a three-dimensional camera and associated electronic processingcircuits. The surgical visualization system 200 is shown being utilizedintraoperatively to identify and facilitate avoidance of certaincritical structures, such as a ureter 201 a and vessels 201 b, in anorgan 203 (a uterus in this embodiment) that are not visible on asurface 205 of the organ 203.

The surgical visualization system 200 is configured to determine anemitter-to-tissue distance d_(e) from an emitter 206 on the surgicaldevice 202 to the surface 205 of the uterus 203 via structured light.The surgical visualization system 200 is configured to extrapolate adevice-to-tissue distance d_(t) from the surgical device 202 to thesurface 205 of the uterus 203 based on the emitter-to-tissue distanced_(e). The surgical visualization system 200 is also configured todetermine a tissue-to-ureter distance d_(A) from the ureter 201 a to thesurface 205 and a camera-to ureter distance d_(w) from the imagingdevice 220 to the ureter 201 a. As described herein, e.g., with respectto the surgical visualization system 100 of FIG. 1 , the surgicalvisualization system 200 is configured to determine the distance d_(w)with spectral imaging and time-of-flight sensors, for example. Invarious embodiments, the surgical visualization system 200 can determine(e.g., triangulate) the tissue-to-ureter distance d_(A) (or depth) basedon other distances and/or the surface mapping logic described herein.

As mentioned above, a surgical visualization system includes a controlsystem configured to control various aspects of the surgicalvisualization system. The control system can have a variety ofconfigurations. FIG. 9 illustrates one embodiment of a control system600 for a surgical visualization system, such as the surgicalvisualization system 100 of FIG. 1 , the surgical visualization system200 of FIG. 8 , or other surgical visualization system described herein.The control system 600 is a conversion system that integrates spectralsignature tissue identification and structured light tissue positioningto identify a critical structure, especially when those structure(s) areobscured by tissue, e.g., by fat, connective tissue, blood tissue,and/or organ(s), and/or by blood, and/or to detect tissue variability,such as differentiating tumors and/or non-healthy tissue from healthytissue within an organ.

The control system 600 is configured for implementing a hyperspectralimaging and visualization system in which a molecular response isutilized to detect and identify anatomy in a surgical field of view. Thecontrol system 600 includes a conversion logic circuit 648 configured toconvert tissue data to usable information for surgeons and/or othermedical practitioners. For example, variable reflectance based onwavelengths with respect to obscuring material can be utilized toidentify the critical structure in the anatomy. Moreover, the controlsystem 600 is configured to combine the identified spectral signatureand the structural light data in an image. For example, the controlsystem 600 can be employed to create of three-dimensional data set forsurgical use in a system with augmentation image overlays. Techniquescan be employed both intraoperatively and preoperatively usingadditional visual information. In various embodiments, the controlsystem 600 is configured to provide warnings to a medical practitionerwhen in the proximity of one or more critical structures. Variousalgorithms can be employed to guide robotic automation andsemi-automated approaches based on the surgical procedure and proximityto the critical structure(s).

A projected array of lights is employed by the control system 600 todetermine tissue shape and motion intraoperatively. Alternatively, flashLidar may be utilized for surface mapping of the tissue.

The control system 600 is configured to detect the critical structure,which as mentioned above can include one or more critical structures,and provide an image overlay of the critical structure and measure thedistance to the surface of the visible tissue and the distance to theembedded/buried critical structure(s). The control system 600 canmeasure the distance to the surface of the visible tissue or detect thecritical structure and provide an image overlay of the criticalstructure.

The control system 600 includes a spectral control circuit 602. Thespectral control circuit 602 can be a field programmable gate array(FPGA) or another suitable circuit configuration, such as theconfigurations described with respect to FIG. 6 , FIG. 7 , and FIG. 8 .The spectral control circuit 602 includes a processor 604 configured toreceive video input signals from a video input processor 606. Theprocessor 604 can be configured for hyperspectral processing and canutilize C/C++ code, for example. The video input processor 606 isconfigured to receive video-in of control (metadata) data such asshutter time, wave length, and sensor analytics, for example. Theprocessor 604 is configured to process the video input signal from thevideo input processor 606 and provide a video output signal to a videooutput processor 608, which includes a hyperspectral video-out ofinterface control (metadata) data, for example. The video outputprocessor 608 is configured to provides the video output signal to animage overlay controller 610.

The video input processor 606 is operatively coupled to a camera 612 atthe patient side via a patient isolation circuit 614. The camera 612includes a solid state image sensor 634. The patient isolation circuit614 can include a plurality of transformers so that the patient isisolated from other circuits in the system. The camera 612 is configuredto receive intraoperative images through optics 632 and the image sensor634. The image sensor 634 can include a CMOS image sensor, for example,or can include another image sensor technology, such as those discussedherein in connection with FIG. 4 . The camera 612 is configured tooutput 613 images in 14 bit/pixel signals. A person skilled in the artwill appreciate that higher or lower pixel resolutions can be employed.The isolated camera output signal 613 is provided to a color RGB fusioncircuit 616, which in this illustrated embodiment employs a hardwareregister 618 and a Nios2 co-processor 620 configured to process thecamera output signal 613. A color RGB fusion output signal is providedto the video input processor 606 and a laser pulsing control circuit622.

The laser pulsing control circuit 622 is configured to control a laserlight engine 624. The laser light engine 624 is configured to outputlight in a plurality of wavelengths (λ1, λ, λ3, . . . λm) including nearinfrared (NIR). The laser light engine 624 can operate in a plurality ofmodes. For example, the laser light engine 624 can operate in two modes.In a first mode, e.g., a normal operating mode, the laser light engine624 is configured to output an illuminating signal. In a second mode,e.g., an identification mode, the laser light engine 624 is configuredto output RGBG and NIR light. In various embodiments, the laser lightengine 624 can operate in a polarizing mode.

Light output 626 from the laser light engine 624 is configured toilluminate targeted anatomy in an intraoperative surgical site 627. Thelaser pulsing control circuit 622 is also configured to control a laserpulse controller 628 for a laser pattern projector 630 configured toproject a laser light pattern 631, such as a grid or pattern of linesand/or dots, at a predetermined wavelength (λ2) on an operative tissueor organ at the surgical site 627. The camera 612 is configured toreceive the patterned light as well as the reflected light outputthrough the camera optics 632. The image sensor 634 is configured toconvert the received light into a digital signal.

The color RGB fusion circuit 616 is also configured to output signals tothe image overlay controller 610 and a video input module 636 forreading the laser light pattern 631 projected onto the targeted anatomyat the surgical site 627 by the laser pattern projector 630. Aprocessing module 638 is configured to process the laser light pattern631 and output a first video output signal 640 representative of thedistance to the visible tissue at the surgical site 627. The data isprovided to the image overlay controller 610. The processing module 638is also configured to output a second video signal 642 representative ofa three-dimensional rendered shape of the tissue or organ of thetargeted anatomy at the surgical site.

The first and second video output signals 640, 642 include datarepresentative of the position of the critical structure on athree-dimensional surface model, which is provided to an integrationmodule 643. In combination with data from the video out processor 608 ofthe spectral control circuit 602, the integration module 643 isconfigured to determine the distance (e.g., distance d_(A) of FIG. 1 )to a buried critical structure (e.g., via triangularization algorithms644), and the distance to the buried critical structure can be providedto the image overlay controller 610 via a video out processor 646. Theforegoing conversion logic can encompass the conversion logic circuit648 intermediate video monitors 652 and the camera 624/laser patternprojector 630 positioned at the surgical site 627.

Preoperative data 650, such as from a CT or MRI scan, can be employed toregister or align certain three-dimensional deformable tissue in variousinstances. Such preoperative data 650 can be provided to the integrationmodule 643 and ultimately to the image overlay controller 610 so thatsuch information can be overlaid with the views from the camera 612 andprovided to the video monitors 652. Embodiments of registration ofpreoperative data are further described in U.S. Pat. Pub. No.2020/0015907 entitled “Integration Of Imaging Data” filed Sep. 11, 2018,which is hereby incorporated by reference herein in its entirety.

The video monitors 652 are configured to output the integrated/augmentedviews from the image overlay controller 610. A medical practitioner canselect and/or toggle between different views on one or more displays. Ona first display 652 a, which is a monitor in this illustratedembodiment, the medical practitioner can toggle between (A) a view inwhich a three-dimensional rendering of the visible tissue is depictedand (B) an augmented view in which one or more hidden criticalstructures are depicted over the three-dimensional rendering of thevisible tissue. On a second display 652 b, which is a monitor in thisillustrated embodiment, the medical practitioner can toggle on distancemeasurements to one or more hidden critical structures and/or thesurface of visible tissue, for example.

The various surgical visualization systems described herein can beutilized to visualize various different types of tissues and/oranatomical structures, including tissues and/or anatomical structuresthat may be obscured from being visualized by EMR in the visible portionof the spectrum. The surgical visualization system can utilize aspectral imaging system, as mentioned above, which can be configured tovisualize different types of tissues based upon their varyingcombinations of constituent materials. In particular, a spectral imagingsystem can be configured to detect the presence of various constituentmaterials within a tissue being visualized based on the absorptioncoefficient of the tissue across various EMR wavelengths. The spectralimaging system can be configured to characterize the tissue type of thetissue being visualized based upon the particular combination ofconstituent materials.

FIG. 10 shows a graph 300 depicting how the absorption coefficient ofvarious biological materials varies across the EMR wavelength spectrum.In the graph 300, the vertical axis 302 represents absorptioncoefficient of the biological material in cm⁻¹, and the horizontal axis304 represents EMR wavelength in μm. A first line 306 in the graph 300represents the absorption coefficient of water at various EMRwavelengths, a second line 308 represents the absorption coefficient ofprotein at various EMR wavelengths, a third line 310 represents theabsorption coefficient of melanin at various EMR wavelengths, a fourthline 312 represents the absorption coefficient of deoxygenatedhemoglobin at various EMR wavelengths, a fifth line 314 represents theabsorption coefficient of oxygenated hemoglobin at various EMRwavelengths, and a sixth line 316 represents the absorption coefficientof collagen at various EMR wavelengths. Different tissue types havedifferent combinations of constituent materials and, therefore, thetissue type(s) being visualized by a surgical visualization system canbe identified and differentiated between according to the particularcombination of detected constituent materials. Accordingly, a spectralimaging system of a surgical visualization system can be configured toemit EMR at a number of different wavelengths, determine the constituentmaterials of the tissue based on the detected absorption EMR absorptionresponse at the different wavelengths, and then characterize the tissuetype based on the particular detected combination of constituentmaterials.

FIG. 11 shows an embodiment of the utilization of spectral imagingtechniques to visualize different tissue types and/or anatomicalstructures. In FIG. 11 , a spectral emitter 320 (e.g., the spectrallight source 150 of FIG. 4 ) is being utilized by an imaging system tovisualize a surgical site 322. The EMR emitted by the spectral emitter320 and reflected from the tissues and/or structures at the surgicalsite 322 is received by an image sensor (e.g., the image sensor 135 ofFIG. 4 ) to visualize the tissues and/or structures, which can be eithervisible (e.g., be located at a surface of the surgical site 322) orobscured (e.g., underlay other tissue and/or structures at the surgicalsite 322). In this embodiment, an imaging system (e.g., the imagingsystem 142 of FIG. 4 ) visualizes a tumor 324, an artery 326, andvarious abnormalities 328 (e.g., tissues not confirming to known orexpected spectral signatures) based upon the spectral signaturescharacterized by the differing absorptive characteristics (e.g.,absorption coefficient) of the constituent materials for each of thedifferent tissue/structure types. The visualized tissues and structurescan be displayed on a display screen associated with or coupled to theimaging system (e.g., the display 146 of the imaging system 142 of FIG.4 ), on a primary display (e.g., the primary display 819 of FIG. 19 ),on a non-sterile display (e.g., the non-sterile displays 807, 809 ofFIG. 19 ), on a display of a surgical hub (e.g., the display of thesurgical hub 806 of FIG. 19 ), on a device/instrument display, and/or onanother display.

The imaging system can be configured to tailor or update the displayedsurgical site visualization according to the identified tissue and/orstructure types. For example, as shown in FIG. 11 , the imaging systemcan display a margin 330 associated with the tumor 324 being visualizedon a display screen associated with or coupled to the imaging system, ona primary display, on a non-sterile display, on a display of a surgicalhub, on a device/instrument display, and/or on another display. Themargin 330 can indicate the area or amount of tissue that should beexcised to ensure complete removal of the tumor 324. A size of themargin 330 can be, for example, in a range of about 5 mm to about 10 mmThe surgical visualization system's control system (e.g., the controlsystem 133 of FIG. 4 ) can be configured to control or update thedimensions of the margin 330 based on the tissues and/or structuresidentified by the imaging system. In this illustrated embodiment, theimaging system has identified multiple abnormalities 328 within thefield of view (FOV). Accordingly, the control system can adjust thedisplayed margin 330 to a first updated margin 332 having sufficientdimensions to encompass the abnormalities 328. Further, the imagingsystem has also identified the artery 326 partially overlapping with theinitially displayed margin 330 (as indicated by a highlighted region 334of the artery 326). Accordingly, the control system can adjust thedisplayed margin to a second updated margin 336 having sufficientdimensions to encompass the relevant portion of the artery 326.

Tissues and/or structures can also be imaged or characterized accordingto their reflective characteristics, in addition to or in lieu of theirabsorptive characteristics described above with respect to FIG. 10 andFIG. 11 , across the EMR wavelength spectrum. For example, FIG. 12 ,FIG. 13 , and FIG. 14 illustrate various graphs of reflectance ofdifferent types of tissues or structures across different EMRwavelengths. FIG. 12 is a graphical representation 340 of anillustrative ureter signature versus obscurants. FIG. 13 is a graphicalrepresentation 342 of an illustrative artery signature versusobscurants. FIG. 14 is a graphical representation 344 of an illustrativenerve signature versus obscurants. The plots in FIG. 12 , FIG. 13 , andFIG. 14 represent reflectance as a function of wavelength (nm) for theparticular structures (ureter, artery, and nerve) relative to thecorresponding reflectances of fat, lung tissue, and blood at thecorresponding wavelengths. These graphs are simply for illustrativepurposes and it should be understood that other tissues and/orstructures could have corresponding detectable reflectance signaturesthat would allow the tissues and/or structures to be identified andvisualized.

Select wavelengths for spectral imaging can be identified and utilizedbased on the anticipated critical structures and/or obscurants at asurgical site (e.g., “selective spectral” imaging). By utilizingselective spectral imaging, the amount of time required to obtain thespectral image can be minimized such that the information can beobtained in real-time and utilized intraoperatively. The wavelengths canbe selected by a medical practitioner or by a control circuit based oninput by a user, e.g., a medical practitioner. In certain instances, thewavelengths can be selected based on machine learning and/or big dataaccessible to the control circuit via, e.g., a cloud or surgical hub.

FIG. 15 illustrates one embodiment of spectral imaging to tissue beingutilized intraoperatively to measure a distance between a waveformemitter and a critical structure that is obscured by tissue. FIG. 15shows an embodiment of a time-of-flight sensor system 404 utilizingwaveforms 424, 425. The time-of-flight sensor system 404 can beincorporated into a surgical visualization system, e.g., as the sensorsystem 104 of the surgical visualization system 100 of FIG. 1 . Thetime-of-flight sensor system 404 includes a waveform emitter 406 and awaveform receiver 408 on the same surgical device 402 (e.g., the emitter106 and the receiver 108 on the same surgical device 102 of FIG. 1 ).The emitted wave 400 extends to a critical structure 401 (e.g., thecritical structure 101 of FIG. 1 ) from the emitter 406, and thereceived wave 425 is reflected back to by the receiver 408 from thecritical structure 401. The surgical device 402 in this illustratedembodiment is positioned through a trocar 410 that extends into a cavity407 in a patient. Although the trocar 410 is used in this in thisillustrated embodiment, other trocars or other access devices can beused, or no access device may be used.

The waveforms 424, 425 are configured to penetrate obscuring tissue 403,such as by having wavelengths in the NIR or SWIR spectrum ofwavelengths. A spectral signal (e.g., hyperspectral, multispectral, orselective spectral) or a photoacoustic signal is emitted from theemitter 406, as shown by a first arrow 407 pointing distally, and canpenetrate the tissue 403 in which the critical structure 401 isconcealed. The emitted waveform 424 is reflected by the criticalstructure 401, as shown by a second arrow 409 pointing proximally. Thereceived waveform 425 can be delayed due to a distance d between adistal end of the surgical device 402 and the critical structure 401.The waveforms 424, 425 can be selected to target the critical structure401 within the tissue 403 based on the spectral signature of thecritical structure 401, as described herein. The emitter 406 isconfigured to provide a binary signal on and off, as shown in FIG. 16 ,for example, which can be measured by the receiver 408.

Based on the delay between the emitted wave 424 and the received wave425, the time-of-flight sensor system 404 is configured to determine thedistance d. A time-of-flight timing diagram 430 for the emitter 406 andthe receiver 408 of FIG. 15 is shown in FIG. 16 . The delay is afunction of the distance d and the distance d is given by:

$d = {\frac{ct}{2} \bullet \frac{q_{2}}{q_{1} + q_{2}}}$

where c=the speed of light; t=length of pulse; q₁=accumulated chargewhile light is emitted; and q₂=accumulated charge while light is notbeing emitted.

The time-of-flight of the waveforms 424, 425 corresponds to the distanced in FIG. 15 . In various instances, additional emitters/receiversand/or pulsing signals from the emitter 406 can be configured to emit anon-penetrating signal. The non-penetrating signal can be configured todetermine the distance from the emitter 406 to the surface 405 of theobscuring tissue 403. In various instances, a depth of the criticalstructure 401 can be determined by:

d _(A) =d _(w) −d _(t)

where d_(A)=the depth of the critical structure 401; d_(w)=the distancefrom the emitter 406 to the critical structure 401 (d in FIG. 15 ); andd_(t),=the distance from the emitter 406 (on the distal end of thesurgical device 402) to the surface 405 of the obscuring tissue 403.

FIG. 17 illustrates another embodiment of a time-of-flight sensor system504 utilizing waves 524 a, 524 b, 524 c, 525 a, 525 b, 525 c is shown.The time-of-flight sensor system 504 can be incorporated into a surgicalvisualization system, e.g., as the sensor system 104 of the surgicalvisualization system 100 of FIG. 1 . The time-of-flight sensor system504 includes a waveform emitter 506 and a waveform receiver 508 (e.g.,the emitter 106 and the receiver 108 of FIG. 1 ). The waveform emitter506 is positioned on a first surgical device 502 a (e.g., the surgicaldevice 102 of FIG. 1 ), and the waveform receiver 508 is positioned on asecond surgical device 502 b. The surgical devices 502 a, 502 b arepositioned through first and second trocars 510 a, 510 b, respectively,which extend into a cavity 507 in a patient. Although the trocars 510 a,510 b are used in this in this illustrated embodiment, other trocars orother access devices can be used, or no access device may be used. Theemitted waves 524 a, 524 b, 524 c extend toward a surgical site from theemitter 506, and the received waves 525 a, 525 b, 525 c are reflectedback to the receiver 508 from various structures and/or surfaces at thesurgical site.

The different emitted waves 524 a, 524 b, 524 c are configured to targetdifferent types of material at the surgical site. For example, the wave524 a targets obscuring tissue 503, the wave 524 b targets a firstcritical structure 501 a (e.g., the critical structure 101 of FIG. 1 ),which is a vessel in this illustrated embodiment, and the wave 524 ctargets a second critical structure 501 b (e.g., the critical structure101 of FIG. 1 ), which is a cancerous tumor in this illustratedembodiment. The wavelengths of the waves 524 a, 524 b, 524 c can be inthe visible light, NIR, or SWIR spectrum of wavelengths. For example,visible light can be reflected off a surface 505 of the tissue 503, andNIR and/or SWIR waveforms can penetrate the surface 505 of the tissue503. In various aspects, as described herein, a spectral signal (e.g.,hyperspectral, multispectral, or selective spectral) or a photoacousticsignal can be emitted from the emitter 506. The waves 524 b, 524 c canbe selected to target the critical structures 501 a, 501 b within thetissue 503 based on the spectral signature of the critical structures501 a, 501 b, as described herein. Photoacoustic imaging is furtherdescribed in various U.S. patent applications, which are incorporated byreference herein in the present disclosure.

The emitted waves 524 a, 524 b, 524 c are reflected off the targetedmaterial, namely the surface 505, the first critical structure 501 a,and the second structure 501 b, respectively. The received waveforms 525a, 525 b, 525 c can be delayed due to distances d_(1a), d_(2a), d_(3a),d_(1b), d_(2b), d_(2c).

In the time-of-flight sensor system 504, in which the emitter 506 andthe receiver 508 are independently positionable (e.g., on separatesurgical devices 502 a, 502 b and/or controlled by separate roboticarms), the various distances d_(1a), d_(2a), d_(3a), d_(1b), d_(2b),d_(2c) can be calculated from the known position of the emitter 506 andthe receiver 508. For example, the positions can be known when thesurgical devices 502 a, 502 b are robotically-controlled. Knowledge ofthe positions of the emitter 506 and the receiver 508, as well as thetime of the photon stream to target a certain tissue and the informationreceived by the receiver 508 of that particular response can allow adetermination of the distances d_(1a), d_(2a), d_(3a), d_(1b), d_(2b),d_(2c). In one aspect, the distance to the obscured critical structures501 a, 501 b can be triangulated using penetrating wavelengths. Becausethe speed of light is constant for any wavelength of visible orinvisible light, the time-of-flight sensor system 504 can determine thevarious distances.

In a view provided to the medical practitioner, such as on a display,the receiver 508 can be rotated such that a center of mass of the targetstructure in the resulting images remains constant, e.g., in a planeperpendicular to an axis of a select target structure 503, 501 a, or 501b. Such an orientation can quickly communicate one or more relevantdistances and/or perspectives with respect to the target structure. Forexample, as shown in FIG. 17 , the surgical site is displayed from aviewpoint in which the critical structure 501 a is perpendicular to theviewing plane (e.g., the vessel is oriented in/out of the page). Such anorientation can be default setting; however, the view can be rotated orotherwise adjusted by a medical practitioner. In certain instances, themedical practitioner can toggle between different surfaces and/or targetstructures that define the viewpoint of the surgical site provided bythe imaging system.

As in this illustrated embodiment, the receiver 508 can be mounted onthe trocar 510 b (or other access device) through which the surgicaldevice 502 b is positioned. In other embodiments, the receiver 508 canbe mounted on a separate robotic arm for which the three-dimensionalposition is known. In various instances, the receiver 508 can be mountedon a movable arm that is separate from a robotic surgical system thatcontrols the surgical device 502 a or can be mounted to an operatingroom (OR) table or fixture that is intraoperatively registerable to therobot coordinate plane. In such instances, the position of the emitter506 and the receiver 508 can be registerable to the same coordinateplane such that the distances can be triangulated from outputs from thetime-of-flight sensor system 504.

Combining time-of-flight sensor systems and near-infrared spectroscopy(NIRS), termed TOF-NIRS, which is capable of measuring the time-resolvedprofiles of NIR light with nanosecond resolution can be found in“Time-Of-Flight Near-Infrared Spectroscopy For NondestructiveMeasurement Of Internal Quality In Grapefruit,” Journal of the AmericanSociety for Horticultural Science, May 2013 vol. 138 no. 3 225-228,which is hereby incorporated by reference in its entirety.

Embodiments of visualization systems and aspects and uses thereof aredescribed further in U.S. Pat. Pub. No. 2020/0015923 entitled “SurgicalVisualization Platform” filed Sep. 11, 2018, U.S. Pat. Pub. No.2020/0015900 entitled “Controlling An Emitter Assembly Pulse Sequence”filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015668 entitled “SingularEMR Source Emitter Assembly” filed Sep. 11, 2018, U.S. Pat. Pub. No.2020/0015925 entitled “Combination Emitter And Camera Assembly” filedSep. 11, 2018, U.S. Pat. Pub. No. 2020/00015899 entitled “SurgicalVisualization With Proximity Tracking Features” filed Sep. 11, 2018,U.S. Pat. Pub. No. 2020/00015903 entitled “Surgical Visualization OfMultiple Targets” filed Sep. 11, 2018, U.S. Pat. No. 10,792,034 entitled“Visualization Of Surgical Devices” filed Sep. 11, 2018, U.S. Pat. Pub.No. 2020/0015897 entitled “Operative Communication Of Light” filed Sep.11, 2018, U.S. Pat. Pub. No. 2020/0015924 entitled “Robotic LightProjection Tools” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015898entitled “Surgical Visualization Feedback System” filed Sep. 11, 2018,U.S. Pat. Pub. No. 2020/0015906 entitled “Surgical Visualization AndMonitoring” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015907entitled “Integration Of Imaging Data” filed Sep. 11, 2018, U.S. Pat.No. 10,925,598 entitled “Robotically-Assisted Surgical Suturing Systems”filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015901 entitled “SafetyLogic For Surgical Suturing Systems” filed Sep. 11, 2018, U.S. Pat. Pub.No. 2020/0015914 entitled “Robotic Systems With Separate PhotoacousticReceivers” filed Sep. 11, 2018, U.S. Pat. Pub. No. 2020/0015902 entitled“Force Sensor Through Structured Light Deflection” filed Sep. 11, 2018,U.S. Pat. Pub. No. 2019/0201136 entitled “Method Of Hub Communication”filed Dec. 4, 2018, U.S. patent application Ser. No. 16/729,772 entitled“Analyzing Surgical Trends By A Surgical System” filed Dec. 30, 2019,U.S. patent application Ser. No. 16/729,747 entitled “Dynamic SurgicalVisualization Systems” filed Dec. 30, 2019, U.S. patent application Ser.No. 16/729,744 entitled “Visualization Systems Using Structured Light”filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,778entitled “System And Method For Determining, Adjusting, And ManagingResection Margin About A Subject Tissue” filed Dec. 30, 2019, U.S.patent application Ser. No. 16/729,729 entitled “Surgical Systems ForProposing And Corroborating Organ Portion Removals” filed Dec. 30, 2019,U.S. patent application Ser. No. 16/729,778 entitled “Surgical SystemFor Overlaying Surgical Instrument Data Onto A Virtual Three DimensionalConstruct Of An Organ” filed Dec. 30, 2019, U.S. patent application Ser.No. 16/729,751 entitled “Surgical Systems For Generating ThreeDimensional Constructs Of Anatomical Organs And Coupling IdentifiedAnatomical Structures Thereto” filed Dec. 30, 2019, U.S. patentapplication Ser. No. 16/729,740 entitled “Surgical Systems CorrelatingVisualization Data And Powered Surgical Instrument Data” filed Dec. 30,2019, U.S. patent application Ser. No. 16/729,737 entitled “AdaptiveSurgical System Control According To Surgical Smoke CloudCharacteristics” filed Dec. 30, 2019, U.S. patent application Ser. No.16/729,796 entitled “Adaptive Surgical System Control According ToSurgical Smoke Particulate Characteristics” filed Dec. 30, 2019, U.S.patent application Ser. No. 16/729,803 entitled “Adaptive VisualizationBy A Surgical System” filed Dec. 30, 2019, U.S. patent application Ser.No. 16/729,807 entitled “Method Of Using Imaging Devices In Surgery”filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,807entitled “Method Of Using Imaging Devices In Surgery” filed Dec. 30,2019, U.S. patent application Ser. No. 17/493,913 entitled “SurgicalMethods Using Fiducial Identification And Tracking” filed on Oct. 5,2021, U.S. patent application Ser. No. 17/493,904 entitled “SurgicalMethods Using Multi-Source Imaging” filed on Oct. 5, 2021, U.S. patentapplication Ser. No. 17/450,020 entitled “Methods And Systems ForControlling Cooperative Surgical Instruments” filed on Oct. 5, 2021,U.S. patent application Ser. No. 17/450,025 entitled “Methods AndSystems For Controlling Cooperative Surgical Instruments With VariableSurgical Site Access Trajectories” filed on Oct. 5, 2021, U.S. patentapplication Ser. No. 17/450,027 entitled “Methods And Systems ForControlling Cooperative Surgical Instruments” filed on Oct. 5, 2021, andU.S. patent application Ser. No. 17/449,765 entitled “CooperativeAccess” filed on Oct. 1, 2021, which are hereby incorporated byreference in their entireties.

Surgical Hubs

The various visualization or imaging systems described herein can beincorporated into a system that includes a surgical hub. In general, asurgical hub can be a component of a comprehensive digital medicalsystem capable of spanning multiple medical facilities and configured toprovide integrated and comprehensive improved medical care to a vastnumber of patients. The comprehensive digital medical system includes acloud-based medical analytics system that is configured to interconnectto multiple surgical hubs located across many different medicalfacilities. The surgical hubs are configured to interconnect with one ormore elements, such as one or more surgical instruments that are used toconduct medical procedures on patients and/or one or more visualizationsystems that are used during performance of medical procedures. Thesurgical hubs provide a wide array of functionality to improve theoutcomes of medical procedures. The data generated by the varioussurgical devices, visualization systems, and surgical hubs about thepatient and the medical procedure may be transmitted to the cloud-basedmedical analytics system. This data may then be aggregated with similardata gathered from many other surgical hubs, visualization systems, andsurgical instruments located at other medical facilities. Variouspatterns and correlations may be found through the cloud-based analyticssystem analyzing the collected data. Improvements in the techniques usedto generate the data may be generated as a result, and theseimprovements may then be disseminated to the various surgical hubs,visualization systems, and surgical instruments. Due to theinterconnectedness of all of the aforementioned components, improvementsin medical procedures and practices may be found that otherwise may notbe found if the many components were not so interconnected.

Examples of surgical hubs configured to receive, analyze, and outputdata, and methods of using such surgical hubs, are further described inU.S. Pat. Pub. No. 2019/0200844 entitled “Method Of Hub Communication,Processing, Storage And Display” filed Dec. 4, 2018, U.S. Pat. Pub. No.2019/0200981 entitled “Method Of Compressing Tissue Within A StaplingDevice And Simultaneously Displaying The Location Of The Tissue WithinThe Jaws” filed Dec. 4, 2018, U.S. Pat. Pub. No. 2019/0201046 entitled“Method For Controlling Smart Energy Devices” filed Dec. 4, 2018, U.S.Pat. Pub. No. 2019/0201114 entitled “Adaptive Control Program UpdatesFor Surgical Hubs” filed Mar. 29, 2018, U.S. Pat. Pub. No. 2019/0201140entitled “Surgical Hub Situational Awareness” filed Mar. 29, 2018, U.S.Pat. Pub. No. 2019/0206004 entitled “Interactive Surgical Systems WithCondition Handling Of Devices And Data Capabilities” filed Mar. 29,2018, U.S. Pat. Pub. No. 2019/0206555 entitled “Cloud-based MedicalAnalytics For Customization And Recommendations To A User” filed Mar.29, 2018, and U.S. Pat. Pub. No. 2019/0207857 entitled “Surgical NetworkDetermination Of Prioritization Of Communication, Interaction, OrProcessing Based On System Or Device Needs” filed Nov. 6, 2018, whichare hereby incorporated by reference in their entireties.

FIG. 18 illustrates one embodiment of a computer-implemented interactivesurgical system 700 that includes one or more surgical systems 702 and acloud-based system (e.g., a cloud 704 that can include a remote server713 coupled to a storage device 705). Each surgical system 702 includesat least one surgical hub 706 in communication with the cloud 704. Inone example, as illustrated in FIG. 18 , the surgical system 702includes a visualization system 708, a robotic system 710, and anintelligent (or “smart”) surgical instrument 712, which are configuredto communicate with one another and/or the hub 706. The intelligentsurgical instrument 712 can include imaging device(s). The surgicalsystem 702 can include an M number of hubs 706, an N number ofvisualization systems 708, an O number of robotic systems 710, and a Pnumber of intelligent surgical instruments 712, where M, N, O, and P areintegers greater than or equal to one that may or may not be equal toany one or more of each other. Various exemplary intelligent surgicalinstruments and robotic systems are described herein.

Data received by a surgical hub from a surgical visualization system canbe used in any of a variety of ways. In an exemplary embodiment, thesurgical hub can receive data from a surgical visualization system inuse with a patient in a surgical setting, e.g., in use in an operatingroom during performance of a surgical procedure. The surgical hub canuse the received data in any of one or more ways, as discussed herein.

The surgical hub can be configured to analyze received data in real timewith use of the surgical visualization system and adjust control one ormore of the surgical visualization system and/or one or more intelligentsurgical instruments in use with the patient based on the analysis ofthe received data. Such adjustment can include, for example, adjustingone or operational control parameters of intelligent surgicalinstrument(s), causing one or more sensors of one or more intelligentsurgical instruments to take a measurement to help gain an understandingof the patient's current physiological condition, and/or currentoperational status of an intelligent surgical instrument, and otheradjustments. Controlling and adjusting operation of intelligent surgicalinstruments is discussed further below. Examples of operational controlparameters of an intelligent surgical instrument include motor speed,cutting element speed, time, duration, level of energy application, andlight emission. Examples of surgical hubs and of controlling andadjusting intelligent surgical instrument operation are describedfurther in previously mentioned U.S. patent application Ser. No.16/729,772 entitled “Analyzing Surgical Trends By A Surgical System”filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,747entitled “Dynamic Surgical Visualization Systems” filed Dec. 30, 2019,U.S. patent application Ser. No. 16/729,744 entitled “VisualizationSystems Using Structured Light” filed Dec. 30, 2019, U.S. patentapplication Ser. No. 16/729,778 entitled “System And Method ForDetermining, Adjusting, And Managing Resection Margin About A SubjectTissue” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,729entitled “Surgical Systems For Proposing And Corroborating Organ PortionRemovals” filed Dec. 30, 2019, U.S. patent application Ser. No.16/729,778 entitled “Surgical System For Overlaying Surgical InstrumentData Onto A Virtual Three Dimensional Construct Of An Organ” filed Dec.30, 2019, U.S. patent application Ser. No. 16/729,751 entitled “SurgicalSystems For Generating Three Dimensional Constructs Of Anatomical OrgansAnd Coupling Identified Anatomical Structures Thereto” filed Dec. 30,2019, U.S. patent application Ser. No. 16/729,740 entitled “SurgicalSystems Correlating Visualization Data And Powered Surgical InstrumentData” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,737entitled “Adaptive Surgical System Control According To Surgical SmokeCloud Characteristics” filed Dec. 30, 2019, U.S. patent application Ser.No. 16/729,796 entitled “Adaptive Surgical System Control According ToSurgical Smoke Particulate Characteristics” filed Dec. 30, 2019, U.S.patent application Ser. No. 16/729,803 entitled “Adaptive VisualizationBy A Surgical System” filed Dec. 30, 2019, and U.S. patent applicationSer. No. 16/729,807 entitled “Method Of Using Imaging Devices InSurgery” filed Dec. 30, 2019, and in U.S. patent application Ser. No.17/068,857 entitled “Adaptive Responses From Smart Packaging Of DrugDelivery Absorbable Adjuncts” filed Oct. 13, 2020, U.S. patentapplication Ser. No. 17/068,858 entitled “Drug Administration DevicesThat Communicate With Surgical Hubs” filed Oct. 13, 2020, U.S. patentapplication Ser. No. 17/068,859 entitled “Controlling Operation Of DrugAdministration Devices Using Surgical Hubs” filed Oct. 13, 2020, U.S.patent application Ser. No. 17/068,863 entitled “Patient MonitoringUsing Drug Administration Devices” filed Oct. 13, 2020, U.S. patentapplication Ser. No. 17/068,865 entitled “Monitoring And CommunicatingInformation Using Drug Administration Devices” filed Oct. 13, 2020, andU.S. patent application Ser. No. 17/068,867 entitled “Aggregating AndAnalyzing Drug Administration Data” filed Oct. 13, 2020, which arehereby incorporated by reference in their entireties.

The surgical hub can be configured to cause visualization of thereceived data to be provided in the surgical setting on a display sothat a medical practitioner in the surgical setting can view the dataand thereby receive an understanding of the operation of the imagingdevice(s) in use in the surgical setting. Such information provided viavisualization can include text and/or images.

FIG. 19 illustrates one embodiment of a surgical system 802 including asurgical hub 806 (e.g., the surgical hub 706 of FIG. 18 or othersurgical hub described herein), a robotic surgical system 810 (e.g., therobotic surgical system 110 of FIG. 1 or other robotic surgical systemherein), and a visualization system 808 (e.g., the visualization system100 of FIG. 1 or other visualization system described herein). Thesurgical hub 806 can be in communication with a cloud, as discussedherein. FIG. 19 shows the surgical system 802 being used to perform asurgical procedure on a patient who is lying down on an operating table814 in a surgical operating room 816. The robotic system 810 includes asurgeon's console 818, a patient side cart 820 (surgical robot), and arobotic system surgical hub 822. The robotic system surgical hub 822 isgenerally configured similar to the surgical hub 822 and can be incommunication with a cloud. In some embodiments, the robotic systemsurgical hub 822 and the surgical hub 806 can be combined. The patientside cart 820 can manipulate an intelligent surgical tool 812 through aminimally invasive incision in the body of the patient while a medicalpractitioner, e.g., a surgeon, nurse, and/or other medical practitioner,views the surgical site through the surgeon's console 818. An image ofthe surgical site can be obtained by an imaging device 824 (e.g., theimaging device 120 of FIG. 1 or other imaging device described herein),which can be manipulated by the patient side cart 820 to orient theimaging device 824. The robotic system surgical hub 822 can be used toprocess the images of the surgical site for subsequent display to thesurgeon through the surgeon's console 818.

A primary display 819 is positioned in the sterile field of theoperating room 816 and is configured to be visible to an operator at theoperating table 814. In addition, as in this illustrated embodiment, avisualization tower 811 can positioned outside the sterile field. Thevisualization tower 811 includes a first non-sterile display 807 and asecond non-sterile display 809, which face away from each other. Thevisualization system 808, guided by the surgical hub 806, is configuredto utilize the displays 807, 809, 819 to coordinate information flow tomedical practitioners inside and outside the sterile field. For example,the surgical hub 806 can cause the visualization system 808 to display asnapshot and/or a video of a surgical site, as obtained by the imagingdevice 824, on one or both of the non-sterile displays 807, 809, whilemaintaining a live feed of the surgical site on the primary display 819.The snapshot and/or video on the non-sterile display 807 and/or 809 canpermit a non-sterile medical practitioner to perform a diagnostic steprelevant to the surgical procedure, for example.

The surgical hub 806 is configured to route a diagnostic input orfeedback entered by a non-sterile medical practitioner at thevisualization tower 811 to the primary display 819 within the sterilefield, where it can be viewed by a sterile medical practitioner at theoperating table 814. For example, the input can be in the form of amodification to the snapshot and/or video displayed on the non-steriledisplay 807 and/or 809, which can be routed to the primary display 819by the surgical hub 806.

The surgical hub 806 is configured to coordinate information flow to adisplay of the intelligent surgical instrument 812, as is described invarious U.S. patent applications that are incorporated by referenceherein in the present disclosure. A diagnostic input or feedback enteredby a non-sterile operator at the visualization tower 818 can be routedby the surgical hub 806 to the display 819 within the sterile field,where it can be viewed by the operator of the surgical instrument 812and/or by other medical practitioner(s) in the sterile field.

The intelligent surgical instrument 812 and the imaging device 824,which is also an intelligent surgical tool, is being used with thepatient in the surgical procedure as part of the surgical system 802.Other intelligent surgical instruments 812 a that can be used in thesurgical procedure, e.g., that can be removably coupled to the patientside cart 820 and be in communication with the robotic surgical system810 and the surgical hub 806, are also shown in FIG. 19 as beingavailable. Non-intelligent (or “dumb”) surgical instruments 817, e.g.,scissors, trocars, cannulas, scalpels, etc., that cannot be incommunication with the robotic surgical system 810 and the surgical hub806 are also shown in FIG. 19 as being available for use.

Operating Intelligent Surgical Instruments

An intelligent surgical device can have an algorithm stored thereon,e.g., in a memory thereof, configured to be executable on board theintelligent surgical device, e.g., by a processor thereof, to controloperation of the intelligent surgical device. In some embodiments,instead of or in addition to being stored on the intelligent surgicaldevice, the algorithm can be stored on a surgical hub, e.g., in a memorythereof, that is configured to communicate with the intelligent surgicaldevice.

The algorithm is stored in the form of one or more sets of pluralitiesof data points defining and/or representing instructions, notifications,signals, etc. to control functions of the intelligent surgical device.In some embodiments, data gathered by the intelligent surgical devicecan be used by the intelligent surgical device, e.g., by a processor ofthe intelligent surgical device, to change at least one variableparameter of the algorithm. As discussed above, a surgical hub can be incommunication with an intelligent surgical device, so data gathered bythe intelligent surgical device can be communicated to the surgical huband/or data gathered by another device in communication with thesurgical hub can be communicated to the surgical hub, and data can becommunicated from the surgical hub to the intelligent surgical device.Thus, instead of or in addition to the intelligent surgical device beingconfigured to change a stored variable parameter, the surgical hub canbe configured to communicate the changed at least one variable, alone oras part of the algorithm, to the intelligent surgical device and/or thesurgical hub can communicate an instruction to the intelligent surgicaldevice to change the at least one variable as determined by the surgicalhub.

The at least one variable parameter is among the algorithm's datapoints, e.g., are included in instructions for operating the intelligentsurgical device, and are thus each able to be changed by changing one ormore of the stored pluralities of data points of the algorithm. Afterthe at least one variable parameter has been changed, subsequentexecution of the algorithm is according to the changed algorithm. Assuch, operation of the intelligent surgical device over time can bemanaged for a patient to increase the beneficial results use of theintelligent surgical device by taking into consideration actualsituations of the patient and actual conditions and/or results of thesurgical procedure in which the intelligent surgical device is beingused. Changing the at least one variable parameter is automated toimprove patient outcomes. Thus, the intelligent surgical device can beconfigured to provide personalized medicine based on the patient and thepatient's surrounding conditions to provide a smart system. In asurgical setting in which the intelligent surgical device is being usedduring performance of a surgical procedure, automated changing of the atleast one variable parameter may allow for the intelligent surgicaldevice to be controlled based on data gathered during the performance ofthe surgical procedure, which may help ensure that the intelligentsurgical device is used efficiently and correctly and/or may help reducechances of patient harm by harming a critical anatomical structure.

The at least one variable parameter can be any of a variety of differentoperational parameters. Examples of variable parameters include motorspeed, motor torque, energy level, energy application duration, tissuecompression rate, jaw closure rate, cutting element speed, loadthreshold, etc.

FIG. 20 illustrates one embodiment of an intelligent surgical instrument900 including a memory 902 having an algorithm 904 stored therein thatincludes at least one variable parameter. The algorithm 904 can be asingle algorithm or can include a plurality of algorithms, e.g.,separate algorithms for different aspects of the surgical instrument'soperation, where each algorithm includes at least one variableparameter. The intelligent surgical instrument 900 can be the surgicaldevice 102 of FIG. 1 , the imaging device 120 of FIG. 1 , the surgicaldevice 202 of FIG. 8 , the imaging device 220 of FIG. 8 , the surgicaldevice 402 of FIG. 15 , the surgical device 502 a of FIG. 17 , thesurgical device 502 b of FIG. 17 , the surgical device 712 of FIG. 18 ,the surgical device 812 of FIG. 19 , the imaging device 824 of FIG. 19 ,or other intelligent surgical instrument. The surgical instrument 900also includes a processor 906 configured to execute the algorithm 904 tocontrol operation of at least one aspect of the surgical instrument 900.To execute the algorithm 904, the processor 906 is configured to run aprogram stored in the memory 902 to access a plurality of data points ofthe algorithm 904 in the memory 902.

The surgical instrument 900 also includes a communications interface908, e.g., a wireless transceiver or other wired or wirelesscommunications interface, configured to communicate with another device,such as a surgical hub 910. The communications interface 908 can beconfigured to allow one-way communication, such as providing data to aremote server (e.g., a cloud server or other server) and/or to a local,surgical hub server, and/or receiving instructions or commands from aremote server and/or a local, surgical hub server, or two-waycommunication, such as providing information, messages, data, etc.regarding the surgical instrument 900 and/or data stored thereon andreceiving instructions, such as from a doctor; a remote server regardingupdates to software; a local, surgical hub server regarding updates tosoftware; etc.

The surgical instrument 900 is simplified in FIG. 20 and can includeadditional components, e.g., a bus system, a handle, a elongate shafthaving an end effector at a distal end thereof, a power source, etc. Theprocessor 906 can also be configured to execute instructions stored inthe memory 902 to control the device 900 generally, including otherelectrical components thereof such as the communications interface 908,an audio speaker, a user interface, etc.

The processor 906 is configured to change at least one variableparameter of the algorithm 904 such that a subsequent execution of thealgorithm 904 will be in accordance with the changed at least onevariable parameter. To change the at least one variable parameter of thealgorithm 904, the processor 906 is configured to modify or update thedata point(s) of the at least one variable parameter in the memory 902.The processor 906 can be configured to change the at least one variableparameter of the algorithm 904 in real time with use of the surgicaldevice 900 during performance of a surgical procedure, which mayaccommodate real time conditions.

Additionally or alternatively to the processor 906 changing the at leastone variable parameter, the processor 906 can be configured to changethe algorithm 904 and/or at least one variable parameter of thealgorithm 904 in response to an instruction received from the surgicalhub 910. In some embodiments, the processor 906 is configured to changethe at least one variable parameter only after communicating with thesurgical hub 910 and receiving an instruction therefrom, which may helpensure coordinated action of the surgical instrument 900 with otheraspects of the surgical procedure in which the surgical instrument 900is being used.

In an exemplary embodiment, the processor 906 executes the algorithm 904to control operation of the surgical instrument 900, changes the atleast one variable parameter of the algorithm 904 based on real timedata, and executes the algorithm 904 after changing the at least onevariable parameter to control operation of the surgical instrument 900.

FIG. 21 illustrates one embodiment of a method 912 of using of thesurgical instrument 900 including a change of at least one variableparameter of the algorithm 904. The processor 906 controls 914 operationof the surgical instrument 900 by executing the algorithm 904 stored inthe memory 902. Based on any of this subsequently known data and/orsubsequently gathered data, the processor 904 changes 916 the at leastone variable parameter of the algorithm 904 as discussed above. Afterchanging the at least one variable parameter, the processor 906 controls918 operation of the surgical instrument 900 by executing the algorithm904, now with the changed at least one variable parameter. The processor904 can change 916 the at least one variable parameter any number oftimes during performance of a surgical procedure, e.g., zero, one, two,three, etc. During any part of the method 912, the surgical instrument900 can communicate with one or more computer systems, e.g., thesurgical hub 910, a remote server such as a cloud server, etc., usingthe communications interface 908 to provide data thereto and/or receiveinstructions therefrom.

Situational Awareness

Operation of an intelligent surgical instrument can be altered based onsituational awareness of the patient. The operation of the intelligentsurgical instrument can be altered manually, such as by a user of theintelligent surgical instrument handling the instrument differently,providing a different input to the instrument, ceasing use of theinstrument, etc. Additionally or alternatively, the operation of anintelligent surgical instrument can be changed automatically by analgorithm of the instrument being changed, e.g., by changing at leastone variable parameter of the algorithm. As mentioned above, thealgorithm can be adjusted automatically without user input requestingthe change. Automating the adjustment during performance of a surgicalprocedure may help save time, may allow medical practitioners to focuson other aspects of the surgical procedure, and/or may ease the processof using the surgical instrument for a medical practitioner, which eachmay improve patient outcomes, such as by avoiding a critical structure,controlling the surgical instrument with consideration of a tissue typethe instrument is being used on and/or near, etc.

The visualization systems described herein can be utilized as part of asituational awareness system that can be embodied or executed by asurgical hub, e.g., the surgical hub 706, the surgical hub 806, or othersurgical hub described herein. In particular, characterizing,identifying, and/or visualizing surgical instruments (including theirpositions, orientations, and actions), tissues, structures, users,and/or other things located within the surgical field or the operatingtheater can provide contextual data that can be utilized by asituational awareness system to infer various information, such as atype of surgical procedure or a step thereof being performed, a type oftissue(s) and/or structure(s) being manipulated by a surgeon or othermedical practitioner, and other information. The contextual data canthen be utilized by the situational awareness system to provide alertsto a user, suggest subsequent steps or actions for the user toundertake, prepare surgical devices in anticipation for their use (e.g.,activate an electrosurgical generator in anticipation of anelectrosurgical instrument being utilized in a subsequent step of thesurgical procedure, etc.), control operation of intelligent surgicalinstruments (e.g., customize surgical instrument operational parametersof an algorithm as discussed further below), and so on.

Although an intelligent surgical device including an algorithm thatresponds to sensed data, e.g., by having at least one variable parameterof the algorithm changed, can be an improvement over a “dumb” devicethat operates without accounting for sensed data, some sensed data canbe incomplete or inconclusive when considered in isolation, e.g.,without the context of the type of surgical procedure being performed orthe type of tissue that is being operated on. Without knowing theprocedural context (e.g., knowing the type of tissue being operated onor the type of procedure being performed), the algorithm may control thesurgical device incorrectly or sub-optimally given the particularcontext-free sensed data. For example, the optimal manner for analgorithm to control a surgical instrument in response to a particularsensed parameter can vary according to the particular tissue type beingoperated on. This is due to the fact that different tissue types havedifferent properties (e.g., resistance to tearing, ease of being cut,etc.) and thus respond differently to actions taken by surgicalinstruments. Therefore, it may be desirable for a surgical instrument totake different actions even when the same measurement for a particularparameter is sensed. As one example, the optimal manner in which tocontrol a surgical stapler in response to the surgical stapler sensingan unexpectedly high force to close its end effector will vary dependingupon whether the tissue type is susceptible or resistant to tearing. Fortissues that are susceptible to tearing, such as lung tissue, thesurgical instrument's control algorithm would optimally ramp down themotor in response to an unexpectedly high force to close to avoidtearing the tissue, e.g., change a variable parameter controlling motorspeed or torque so the motor is slower. For tissues that are resistantto tearing, such as stomach tissue, the instrument's algorithm wouldoptimally ramp up the motor in response to an unexpectedly high force toclose to ensure that the end effector is clamped properly on the tissue,e.g., change a variable parameter controlling motor speed or torque sothe motor is faster. Without knowing whether lung or stomach tissue hasbeen clamped, the algorithm may be sub-optimally changed or not changedat all.

A surgical hub can be configured to derive information about a surgicalprocedure being performed based on data received from various datasources and then control modular devices accordingly. In other words,the surgical hub can be configured to infer information about thesurgical procedure from received data and then control the modulardevices operably coupled to the surgical hub based upon the inferredcontext of the surgical procedure. Modular devices can include anysurgical device that is controllable by a situational awareness system,such as visualization system devices (e.g., a camera, a display screen,etc.), smart surgical instruments (e.g., an ultrasonic surgicalinstrument, an electrosurgical instrument, a surgical stapler, smokeevacuators, scopes, etc.). A modular device can include sensor(s)configured to detect parameters associated with a patient with which thedevice is being used and/or associated with the modular device itself.

The contextual information derived or inferred from the received datacan include, for example, a type of surgical procedure being performed,a particular step of the surgical procedure that the surgeon (or othermedical practitioner) is performing, a type of tissue being operated on,or a body cavity that is the subject of the surgical procedure. Thesituational awareness system of the surgical hub can be configured toderive the contextual information from the data received from the datasources in a variety of different ways. In an exemplary embodiment, thecontextual information received by the situational awareness system ofthe surgical hub is associated with a particular control adjustment orset of control adjustments for one or more modular devices. The controladjustments each correspond to a variable parameter. In one example, thesituational awareness system includes a pattern recognition system, ormachine learning system (e.g., an artificial neural network), that hasbeen trained on training data to correlate various inputs (e.g., datafrom databases, patient monitoring devices, and/or modular devices) tocorresponding contextual information regarding a surgical procedure. Inother words, a machine learning system can be trained to accuratelyderive contextual information regarding a surgical procedure from theprovided inputs. In another example, the situational awareness systemcan include a lookup table storing pre-characterized contextualinformation regarding a surgical procedure in association with one ormore inputs (or ranges of inputs) corresponding to the contextualinformation. In response to a query with one or more inputs, the lookuptable can return the corresponding contextual information for thesituational awareness system for controlling at least one modulardevice. In another example, the situational awareness system includes afurther machine learning system, lookup table, or other such system,which generates or retrieves one or more control adjustments for one ormore modular devices when provided the contextual information as input.

A surgical hub including a situational awareness system may provide anynumber of benefits for a surgical system. One benefit includes improvingthe interpretation of sensed and collected data, which would in turnimprove the processing accuracy and/or the usage of the data during thecourse of a surgical procedure. Another benefit is that the situationalawareness system for the surgical hub may improve surgical procedureoutcomes by allowing for adjustment of surgical instruments (and othermodular devices) for the particular context of each surgical procedure(such as adjusting to different tissue types) and validating actionsduring a surgical procedure. Yet another benefit is that the situationalawareness system may improve surgeon's and/or other medicalpractitioners' efficiency in performing surgical procedures byautomatically suggesting next steps, providing data, and adjustingdisplays and other modular devices in the surgical theater according tothe specific context of the procedure. Another benefit includesproactively and automatically controlling modular devices according tothe particular step of the surgical procedure that is being performed toreduce the number of times that medical practitioners are required tointeract with or control the surgical system during the course of asurgical procedure, such as by a situationally aware surgical hubproactively activating a generator to which an RF electrosurgicalinstrument is connected if it determines that a subsequent step of theprocedure requires the use of the instrument. Proactively activating theenergy source allows the instrument to be ready for use a soon as thepreceding step of the procedure is completed.

For example, a situationally aware surgical hub can be configured todetermine what type of tissue is being operated on. Therefore, when anunexpectedly high force to close a surgical instrument's end effector isdetected, the situationally aware surgical hub can be configured tocorrectly ramp up or ramp down a motor of the surgical instrument forthe type of tissue, e.g., by changing or causing change of at least onevariable parameter of an algorithm for the surgical instrument regardingmotor speed or torque.

For another example, a type of tissue being operated can affectadjustments that are made to compression rate and load thresholds of asurgical stapler for a particular tissue gap measurement. Asituationally aware surgical hub can be configured to infer whether asurgical procedure being performed is a thoracic or an abdominalprocedure, allowing the surgical hub to determine whether the tissueclamped by an end effector of the surgical stapler is lung tissue (for athoracic procedure) or stomach tissue (for an abdominal procedure). Thesurgical hub can then be configured to cause adjustment of thecompression rate and load thresholds of the surgical staplerappropriately for the type of tissue, e.g., by changing or causingchange of at least one variable parameter of an algorithm for thesurgical stapler regarding compression rate and load threshold.

As yet another example, a type of body cavity being operated in duringan insufflation procedure can affect the function of a smoke evacuator.A situationally aware surgical hub can be configured to determinewhether the surgical site is under pressure (by determining that thesurgical procedure is utilizing insufflation) and determine theprocedure type. As a procedure type is generally performed in a specificbody cavity, the surgical hub can be configured to control a motor rateof the smoke evacuator appropriately for the body cavity being operatedin, e.g., by changing or causing change of at least one variableparameter of an algorithm for the smoke evacuator regarding motor rate.Thus, a situationally aware surgical hub may provide a consistent amountof smoke evacuation for both thoracic and abdominal procedures.

As yet another example, a type of procedure being performed can affectthe optimal energy level for an ultrasonic surgical instrument or radiofrequency (RF) electrosurgical instrument to operate at. Arthroscopicprocedures, for example, require higher energy levels because an endeffector of the ultrasonic surgical instrument or RF electrosurgicalinstrument is immersed in fluid. A situationally aware surgical hub canbe configured to determine whether the surgical procedure is anarthroscopic procedure. The surgical hub can be configured to adjust anRF power level or an ultrasonic amplitude of the generator (e.g., adjustenergy level) to compensate for the fluid filled environment, e.g., bychanging or causing change of at least one variable parameter of analgorithm for the instrument and/or a generator regarding energy level.Relatedly, a type of tissue being operated on can affect the optimalenergy level for an ultrasonic surgical instrument or RF electrosurgicalinstrument to operate at. A situationally aware surgical hub can beconfigured to determine what type of surgical procedure is beingperformed and then customize the energy level for the ultrasonicsurgical instrument or RF electrosurgical instrument, respectively,according to the expected tissue profile for the surgical procedure,e.g., by changing or causing change of at least one variable parameterof an algorithm for the instrument and/or a generator regarding energylevel. Furthermore, a situationally aware surgical hub can be configuredto adjust the energy level for the ultrasonic surgical instrument or RFelectrosurgical instrument throughout the course of a surgicalprocedure, rather than just on a procedure-by-procedure basis. Asituationally aware surgical hub can be configured to determine whatstep of the surgical procedure is being performed or will subsequentlybe performed and then update the control algorithm(s) for the generatorand/or ultrasonic surgical instrument or RF electrosurgical instrumentto set the energy level at a value appropriate for the expected tissuetype according to the surgical procedure step.

As another example, a situationally aware surgical hub can be configuredto determine whether the current or subsequent step of a surgicalprocedure requires a different view or degree of magnification on adisplay according to feature(s) at the surgical site that the surgeonand/or other medical practitioner is expected to need to view. Thesurgical hub can be configured to proactively change the displayed view(supplied by, e.g., an imaging device for a visualization system)accordingly so that the display automatically adjusts throughout thesurgical procedure.

As yet another example, a situationally aware surgical hub can beconfigured to determine which step of a surgical procedure is beingperformed or will subsequently be performed and whether particular dataor comparisons between data will be required for that step of thesurgical procedure. The surgical hub can be configured to automaticallycall up data screens based upon the step of the surgical procedure beingperformed, without waiting for the surgeon or other medical practitionerto ask for the particular information.

As another example, a situationally aware surgical hub can be configuredto determine whether a surgeon and/or other medical practitioner ismaking an error or otherwise deviating from an expected course of actionduring the course of a surgical procedure, e.g., as provided in apre-operative surgical plan. For example, the surgical hub can beconfigured to determine a type of surgical procedure being performed,retrieve a corresponding list of steps or order of equipment usage(e.g., from a memory), and then compare the steps being performed or theequipment being used during the course of the surgical procedure to theexpected steps or equipment for the type of surgical procedure that thesurgical hub determined is being performed. The surgical hub can beconfigured to provide an alert (visual, audible, and/or tactile)indicating that an unexpected action is being performed or an unexpecteddevice is being utilized at the particular step in the surgicalprocedure.

In certain instances, operation of a robotic surgical system, such asany of the various robotic surgical systems described herein, can becontrolled by the surgical hub based on its situational awareness and/orfeedback from the components thereof and/or based on information from acloud (e.g., the cloud 713 of FIG. 18 ).

Embodiments of situational awareness systems and using situationalawareness systems during performance of a surgical procedure aredescribed further in previously mentioned U.S. patent application Ser.No. 16/729,772 entitled “Analyzing Surgical Trends By A Surgical System”filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,747entitled “Dynamic Surgical Visualization Systems” filed Dec. 30, 2019,U.S. patent application Ser. No. 16/729,744 entitled “VisualizationSystems Using Structured Light” filed Dec. 30, 2019, U.S. patentapplication Ser. No. 16/729,778 entitled “System And Method ForDetermining, Adjusting, And Managing Resection Margin About A SubjectTissue” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,729entitled “Surgical Systems For Proposing And Corroborating Organ PortionRemovals” filed Dec. 30, 2019, U.S. patent application Ser. No.16/729,778 entitled “Surgical System For Overlaying Surgical InstrumentData Onto A Virtual Three Dimensional Construct Of An Organ” filed Dec.30, 2019, U.S. patent application Ser. No. 16/729,751 entitled “SurgicalSystems For Generating Three Dimensional Constructs Of Anatomical OrgansAnd Coupling Identified Anatomical Structures Thereto” filed Dec. 30,2019, U.S. patent application Ser. No. 16/729,740 entitled “SurgicalSystems Correlating Visualization Data And Powered Surgical InstrumentData” filed Dec. 30, 2019, U.S. patent application Ser. No. 16/729,737entitled “Adaptive Surgical System Control According To Surgical SmokeCloud Characteristics” filed Dec. 30, 2019, U.S. patent application Ser.No. 16/729,796 entitled “Adaptive Surgical System Control According ToSurgical Smoke Particulate Characteristics” filed Dec. 30, 2019, U.S.patent application Ser. No. 16/729,803 entitled “Adaptive VisualizationBy A Surgical System” filed Dec. 30, 2019, and U.S. patent applicationSer. No. 16/729,807 entitled “Method Of Using Imaging Devices InSurgery” filed Dec. 30, 2019.

Surgical Procedures of the Lung

Various aspects of the devices, systems, and methods described hereinmay relate to a surgical procedure performed on a lung. For example, alung resection, e.g., a lobectomy, is a surgical procedure in which allor part, e.g., one or more lobes, of a lung is removed. The purpose ofperforming a lung resection is to treat a damaged or diseased lung as aresult of, for example, lung cancer, emphysema, or bronchiectasis.

During a lung resection, the lung or lungs are first deflated, andthereafter one or more incisions are made on the patient's side betweenthe patient's ribs to reach the lungs laparoscopically. Surgicalinstruments, such as graspers and a laparoscope, are inserted throughthe incision. Once the infected or damaged area of the lung isidentified, the area is dissected from the lung and removed from the oneor more incisions. The dissected area and the one or more incisions canbe closed, for example, with a surgical stapler or stitches.

Since the lung is deflated during surgery, the lung, or certain portionsthereof, may need to be mobilized to allow the surgical instruments toreach the surgical site. This mobilization can be carried out bygrasping the outer tissue layer of the lung with graspers and applying aforce to the lung through the graspers. However, the pleura andparenchyma of the lung are very fragile and therefore can be easilyripped or torn under the applied force. Additionally, duringmobilization, the graspers can cut off blood supply to one or more areasof the lung.

Further, a breathing tube is placed into the patient's airway to alloweach lung to be separately inflated during surgery. Inflation of thelung can cause the lung to move and match pre-operative imaging and/orallow the surgeon to check for leaks at the dissected area(s). However,by inflating the whole lung, working space is lost around the lung dueto the filling of the thoracic cavity. Additionally, inflating a wholelung can take time and does not guarantee easy leak detection ifmultiple portions of the lung are operated on during the surgicalprocedure.

Surgical Procedures of the Colon

Various aspects of the devices, systems, and methods described hereinmay relate to a surgical procedure performed on a colon. For example,surgery is often the main treatment for early-stage colon cancers. Thetype of surgery used depends on the stage (extent) of the cancer, whereit is in the colon, and the goal of the surgery. Some early coloncancers (stage 0 and some early stage I tumors) and most polyps can beremoved during a colonoscopy. However, if the cancer has progressed, alocal excision or colectomy may be required. A colectomy is surgery toremove all or part of the colon. In certain instances, nearby lymphnodes are also removed. If only part of the colon is removed, it'scalled a hemicolectomy, partial colectomy, or segmental resection inwhich the surgeon takes out the diseased part of the colon with a smallsegment of non-diseased colon on either side. Usually, about one-fourthto one-third of the colon is removed, depending on the size and locationof the cancer. Major resections of the colon are illustrated in FIG. 22, in which A-B is a right hemicolectomy, A-C is an extended righthemicolectomy, B-C is a transverse colectomy, C-E is a lefthemicolectomy, D-E is a sigmoid colectomy, D-F is an anterior resection,D-G is a (ultra) low anterior resection, D-H is an abdomino-perinealresection, A-D is a subtotal colectomy, A-E is a total colectomy, andA-H is a total procto-colectomy. Once the resection is complete, theremaining intact sections of colon are then reattached.

A colectomy can be performed through an open colectomy, where a singleincision through the abdominal wall is used to access the colon forseparation and removal of the affected colon tissue, and through alaparoscopic-assisted colectomy. With a laparoscopic-assisted colectomy,the surgery is done through many smaller incisions with surgicalinstruments and a laparoscope passing through the small incisions toremove the entire colon or a part thereof. At the beginning of theprocedure, the abdomen is inflated with gas, e.g., carbon dioxide, toprovide a working space for the surgeon. The laparoscope transmitsimages inside the abdominal cavity, giving the surgeon a magnified viewof the patient's internal organs on a monitor or other display. Severalother cannulas are inserted to allow the surgeon to work inside andremove part(s) of the colon. Once the diseased parts of the colon areremoved, the remaining ends of the colon are attached to each other,e.g., via staplers or stitches. The entire procedure may be completedthrough the cannulas or by lengthening one of the small cannulaincisions.

During a laparoscopic-assisted colectomy procedure, it is oftendifficult to obtain an adequate operative field. Oftentimes, dissectionsare made deep in the pelvis which makes it difficult to obtain adequatevisualization of the area. As a result, the lower rectum must be liftedand rotated to gain access to the veins and arteries around both sidesof the rectum during mobilization. During manipulation of the lowerrectum, bunching of tissue and/or overstretching of tissue can occur.Additionally, a tumor within the rectum can cause adhesions in thesurrounding pelvis, and as a result, this can require freeing the rectalstump and mobilizing the mesentery and blood supply before transectionand removal of the tumor.

Further, multiple graspers are needed to position the tumor for removalfrom the colon. During dissection of the colon, the tumor should beplaced under tension, which requires grasping and stretching thesurrounding healthy tissue of the colon. However, the manipulating ofthe tissue surrounding the tumor can suffer from reduced blood flow andtrauma due to the graspers placing a high grip force on the tissue.Additionally, during a colectomy, the transverse colon and upperdescending colon may need to be mobilized to allow the healthy, goodremaining colon to be brought down to connect to the rectal stump afterthe section of the colon containing the tumor is transected and removed.

After a colectomy, the remaining healthy portions of the colon must bereattached to one another to create a path for waste to leave the body.However, when using laparoscopic instruments to perform the colectomy,one single entry port may not have a large enough range of motion tomove the one end of the colon to a connecting portion of the colon. Assuch, a second entry port is therefore needed to laparoscopically insertsurgical instruments to help mobilize the colon in order to properlyposition the colon.

Surgical Procedures of the Stomach

Various aspects of the devices, systems, and methods described hereinmay relate to a surgical procedure performed on a stomach. For example,surgery is the most common treatment for stomach cancer. When surgery isrequired for stomach cancer, the goal is to remove the entire tumor aswell as a good margin of healthy stomach tissue around the tumor.Different procedures can be used to remove stomach cancer. The type ofprocedure used depends on what part of the stomach the cancer is locatedand how far it has grown into nearby areas. For example, endoscopicmucosal resection (EMR) and endoscopic submucosal dissection (ESD) areprocedures on the stomach can be used to treat some early-stage cancers.These procedures do not require a cut in the skin, but instead thesurgeon passes an endoscope down the throat and into the stomach of thepatient. Surgical tools (e.g., MEGADYNE™ Tissue Dissector orElectrosurgical Pencils) are then passed through the working channel ofthe endoscope to remove the tumor and some layers of the normal stomachwall below and around it.

Other surgical procedures performed on a stomach include a subtotal(partial) or a total gastrectomy that can be performed as an openprocedure. e.g., surgical instruments are inserted through a largeincision in the skin of the abdomen, or as a laparoscopic procedure,e.g., surgical instruments are inserted into the abdomen through severalsmall cuts. For example, a laparoscopic gastrectomy procedure generallyinvolves insufflation of the abdominal cavity with carbon dioxide gas toa pressure of around 15 millimeters of mercury (mm Hg). The abdominalwall is pierced and a straight tubular cannula or trocar, such as acannula or trocar having a diameter in a range of about 5 mm to about 10mm, is then inserted into the abdominal cavity. A laparoscope connectedto an operating room monitor is used to visualize the operative fieldand is placed through one of the trocar(s). Laparoscopic surgicalinstruments are placed through two or more additional cannulas ortrocars for manipulation by medical practitioner(s), e.g., surgeon andsurgical assistant(s), to remove the desired portion(s) of the stomach.

In certain instances, laparoscopic and endoscopic cooperative surgerycan be used to remove gastric tumors. This cooperative surgery typicallyinvolves introduction of an endoscope, e.g., a gastroscope, andlaparoscopic trocars. A laparoscope and tissue manipulation anddissection surgical instruments are introduced through the trocar. Thetumor location can be identified via the endoscope and a cutting elementthat is inserted into the working channel of the endoscope is then usedfor submucosal resection around the tumor. A laparoscopic dissectionsurgical instrument is then used for seromuscular dissection adjacentthe tumor margins to create an incision through the stomach wall. Thetumor is then pivoted through this incision from the intraluminal space,e.g., inside the stomach, to the extraluminal space, e.g., outside ofthe stomach. A laparoscopic surgical instrument, e.g., an endocutter,can be used to then complete the transection of the tumor from thestomach wall and seal the incision.

Surgical Procedures of the Intestine

Various aspects of the devices, systems, and methods described hereinmay relate to a surgical procedure performed on an intestine. Forexample, a duodenal mucosal resurfacing (DMR) procedure can be performedendoscopically to treat insulin-resistant metabolic diseases such astype 2 diabetes. The DMR procedure can be an effective treatment becauseit affects detection of food. The DMR procedure inhibits duodenumfunction such that food tends to be sensed deeper in the intestine thannormal, e.g., sensed after passage through the duodenum (which is thefirst part of the small intestine). The patient's body thus senses sugardeeper in the intestine than is typical and thus reacts to the sugarlater than is typical such that glycemic control can be improved. Theirregular function of the duodenum changes the body's typical responseto the food and, through nervous system and chemical signals, causes thebody to adapt its response to the glucose level to increase insulinlevels.

In the DMR procedure, the duodenal mucosa is lifted, such as withsaline, and then the mucosa is ablated, e.g., using an ablation deviceadvanced into the duodenum through a working channel of an endoscope.Lifting the mucosa before ablation helps protect the duodenum's outerlayers from being damaged by the ablation. After the mucosa is ablated,the mucosa later regenerates. Examples of ablation devices are NeuWave™ablation probes (available from Ethicon US LLC of Cincinnati, Ohio).Another example of an ablation device is the Hyblate catheter ablationprobe (available from Hyblate Medical of Misgav, Israel). Anotherexample of an ablation device is the Barxx™ HaloFlex (available fromMedtronic of Minneapolis, Minn.).

FIG. 23 illustrates one embodiment of a DMR procedure. As shown in FIG.23 , a laparoscope 1400 is positioned external to a duodenum 1402 forexternal visualization of the duodenum 1402. An endoscope 1404 isadvanced transorally through an esophagus 1406, through a stomach 1408,and into the duodenum 1402 for internal visualization of the duodenum1402. An ablation device 1410 is advanced through a working channel ofthe endoscope 1404 to extend distally from the endoscope 1404 into theduodenum 1402. A balloon 1412 of the ablation device 1410 is shownexpanded or inflated in FIG. 23 . The expanded or inflated balloon 1412can help center the ablation device's electrode so even circumferentialablating can occur before the ablation device 1410 is advanced and/orretracted to repeat ablation. Before the mucosa is ablated using theablation device 1410, the duodenal mucosa is lifted, such as withsaline. In some embodiments in addition to or instead of including theballoon 1412, the ablation device 1410 can be expandable/collapsibleusing an electrode array or basket configured to expand and collapse.

The laparoscope's external visualization of the duodenum 1402 can allowfor thermal monitoring of the duodenum 1402, which may help ensure thatthe outer layers of the duodenum 1402 are not damaged by the ablation ofthe duodenal mucosa, such as by the duodenum being perforated. Variousembodiments of thermal monitoring are discussed further, for example,below and in U.S. patent application Ser. No. 17/493,904 entitled“Surgical Methods Using Multi-Source Imaging” filed on Oct. 5, 2021. Theendoscope 1404 and/or the ablation device 1410 can include a fiducialmarker thereon that the laparoscope 1400 can be configured to visualizethrough the duodenum's tissue, e.g., by using invisible light, to helpdetermine where the laparoscope 1400 should externally visualize theduodenum 1402 at a location where ablation occurs. Various embodimentsof fiducial markers are discussed further, for example, below and inU.S. patent application Ser. No. 17/493,913 entitled “Surgical MethodsUsing Fiducial Identification And Tracking” filed on Oct. 5, 2021.

Intraluminal and Extraluminal Cooperation

Devices, systems, and methods for multi-source imaging provided hereinmay allow for intraluminal and extraluminal cooperation. In general, inintraluminal and extraluminal cooperation, a hollow organ or body lumenis visualized from an internal point of view (intraluminalvisualization) and is visualized from an external point of view(extraluminal visualization). The intraluminal and extraluminalvisualizations cooperate to provide a medical practitioner a morecomplete view of the hollow organ or body lumen at least at an area ofinterest thereof during performance of a surgical procedure than wouldbe possible if only one of intraluminal and extraluminal was available.The medical practitioner may therefore be able to make more informeddecisions in performing the surgical procedure and/or a controller(e.g., a controller of a surgical hub, a robotic surgical system, orother computer system) may be able to effect better control of surgicalinstruments and/or imaging devices.

A DMR procedure is an example of a surgical procedure that can includeintraluminal and extraluminal cooperation. In a DMR procedure in whichan endoscope and a laparoscope are used to visualize inside and outsidea duodenum, respectively, blood flow intervention can be provided fromthe laparoscopic side of the duodenum, e.g., from outside the duodenum.The blood flow intervention may reinforce the endoluminal treatment tominimize mucosal recovery and prolong durability of the effects of themucosal ablation. A surgical implant introduced laparoscopically can beconfigured to provide the blood flow intervention. The surgical implantcan be configured to guide effects of the mucosal ablation and/or tomarginalize blood supply to the duodenal region being ablated.

Reduced vascular flow can be measured, for example, by an infrared (IR)reading of a contrast agent, such as indocyanine green (ICG) or othercontrast agent, introduced into the patient's blood. The laparoscopepositioned outside the duodenum can be configured to visualize using IR(and possibly one or more additional visualization modalities, such asvisual light, UV, etc.), thereby allowing the laparoscope to provide theIR reading of the contrast agent.

The reinforcement of the endoluminal treatment may be used to adapt theinterconnection of the pancreas and the small intestine minorly tochange gastrointestinal motility by allowing fatty acids to travelfaster through the intestines. This laparoscopic adaption can be guidedand collaborated with via small restrictions or increased structureopening or connections by the endoscope.

For another example, in a DMR procedure in which an endoscope and alaparoscope are used to visualize inside and outside a duodenum, anadjunct can be implanted at the duodenum to improve therapeutic effectof the DMR procedure. The adjunct is a medicant-eluting adjunct, whichallows the adjunct to provide treatment to the duodenum from outside theduodenum to help the duodenum heal properly after the ablation. Themedicant eluted by the adjunct can be configured to limit sensing in theduodenum and thereby prevent signal transmission of the sensing toanother part of the patient's body. The mucosal ablation and the adjunctmay thus each contribute to the DMR procedure's therapeutic effect.

The adjunct can releasably retain therein at least one medicant that canbe selected from a large number of different medicants. Medicantsinclude, but are not limited to, drugs or other agents included within,or associated with, the adjunct that have a desired functionality.Examples of medicants include antimicrobial agents such as antibacterialand antibiotic agents, antifungal agents, antiviral agents,anti-inflammatory agents, growth factors, analgesics, anesthetics,tissue matrix degeneration inhibitors, anti-cancer agents, hemostaticagents, and other agents that elicit a biological response.

Examples of antimicrobial agents include Ionic Silver, Aminoglycosides,Streptomycin, Polypeptides, Bacitracin, Triclosan, Tetracyclines,Doxycycline, Minocycline, Demeclocycline, Tetracycline, Oxytetracycline,Chloramphenicol, Nitrofurans, Furazolidone, Nitrofurantoin,Beta-lactams, Penicillins, Amoxicillin, Amoxicillin+, Clavulanic Acid,Azlocillin, Flucloxacillin, Ticarcillin, Piperacillin+tazobactam,Tazocin, Biopiper TZ, Zosyn, Carbapenems, Imipenem, Meropenem,Ertapenem, Doripenem, Biapenem, Panipenem/betamipron, Quinolones,Ciprofloxacin, Enoxacin, Gatifloxacin, Gemifloxacin, Levofloxacin,Lomefloxacin, Moxifloxacin, Nalidixic Acid, Norfloxacin, Sulfonamides,Mafenide, Sulfacetamide, Sulfadiazine, Silver Sulfadiazine,Sulfadimethoxine, Sulfamethizole, Sulfamethoxazole, Sulfasalazine,Sulfisoxazole, Bactrim, Prontosil, Ansamycins, Geldanamycin, Herbimycin,Fidaxomicin, Glycopeptides, Teicoplanin, Vancomycin, Telavancin,Dalbavancin, Oritavancin, Lincosamides, Clindamycin, Lincomycin,Lipopeptide, Daptomycin, Macrolides, Azithromycin, Clarithromycin,Erythromycin, Roxithromycin, Telithromycin, Spiramycin, Oxazolidinones,Linezolid, Aminoglycosides, Amikacin, Gentamicin, Kanamycin, Neomycin,Netilmicin, Tobramycin, Paromycin, Paromomycin, Cephalosporins,Ceftobiprole, Ceftolozane, Cefclidine, Flomoxef, Monobactams, Aztreonam,Colistin, and Polymyxin B.

Examples of antifungal agents include Triclosan, Polyenes, AmphotericinB, Candicidin, Filipin, Hamycin, Natamycin, Nystatin, Rimocidin, Azoles,Imidazole, Triazole, Thiazole, Allylamines, Amorolfin, Butenafine,Naftifine, Terbinafine, Echinocandins, Anidulafungin, Caspofungin,Micafungin, Ciclopirox, and Benzoic Acid.

Examples of antiviral agents include uncoating inhibitors such as, forexample, Amantadine, Rimantadine, Pleconaril; reverse transcriptaseinhibitors such as, for example, Acyclovir, Lamivudine, Antisenses,Fomivirsen, Morpholinos, Ribozymes, Rifampicin; and virucidals such as,for example, Cyanovirin-N, Griffithsin, Scytovirin, α-Lauroyl-L-arginineethyl ester (LAE), and Ionic Silver.

Examples of anti-inflammatory agents include non-steroidalanti-inflammatory agents (e.g., Salicylates, Aspirin, Diflunisal,Propionic Acid Derivatives, Ibuprofen, Naproxen, Fenoprofen, andLoxoprofen), acetic acid derivatives (e.g., Tolmetin, Sulindac, andDiclofenac), enolic acid derivatives (e.g., Piroxicam, Meloxicam,Droxicam, and Lornoxicam), anthranilic acid derivatives (e.g., MefenamicAcid, Meclofenamic Acid, and Flufenamic Acid), selective COX-2inhibitors (e.g., Celecoxib (Celebrex), Parecoxib, Rofecoxib (Vioxx),Sulfonanilides, Nimesulide, and Clonixin), immune selectiveanti-inflammatory derivatives, corticosteroids (e.g., Dexamethasone),and iNOS inhibitors.

Examples of growth factors include those that are cell signalingmolecules that stimulate cell growth, healing, remodeling,proliferation, and differentiation. Exemplary growth factors can beshort-ranged (paracrine), long ranged (endocrine), or self-stimulating(autocrine). Further examples of the growth factors include growthhormones (e.g., a recombinant growth factor, Nutropin, Humatrope,Genotropin, Norditropin, Saizen, Omnitrope, and a biosynthetic growthfactor), Epidermal Growth Factor (EGF) (e.g., inhibitors, Gefitinib,Erlotinib, Afatinib, and Cetuximab), heparin-binding EGF like growthfactors (e.g., Epiregulin, Betacellulin, Amphiregulin, and Epigen),Transforming Growth Factor alpha (TGF-a), Neuroregulin 1-4, FibroblastGrowth Factors (FGFs) (e.g., FGF1-2, FGF2, FGF11-14, FGF18, FGF15/19,FGF21, FGF23, FGF7 or Keratinocyte Growth Factor (KGF), FGF10 or KGF2,and Phenytoin), Insuline-like Growth Factors (IGFs) (e.g., IGF-1, IGF-2,and Platelet Derived Growth Factor (PDGF)), Vascular Endothelial GrowthFactors (VEGFs) (e.g., inhibitors, Bevacizumab, Ranibizumab, VEGF-A,VEGF-B, VEGF-C, VEGF-D and Becaplermin).

Additional examples of the growth factors include cytokines, such asGranulocyte Macrophage Colony Stimulating Factors (GM-CSFs) (e.g.,inhibitors that inhibit inflammatory responses, and GM-CSF that has beenmanufactured using recombinant DNA technology and via recombinantyeast-derived sources), Granulocyte Colony Stimulating Factors (G-CSFs)(e.g., Filgrastim, Lenograstim, and Neupogen), Tissue Growth Factor Beta(TGF-B), Leptin, and interleukins (ILs) (e.g., IL-1a, IL-1b,Canakinumab, IL-2, Aldesleukin, Interking, Denileukin Diftitox, IL-3,IL-6, IL-8, IL-10, IL-11, and Oprelvekin). Examples of the growthfactors further include erythropoietin (e.g., Darbepoetin, Epocept,Dynepo, Epomax, NeoRecormon, Silapo, and Retacrit).

Examples of analgesics include Narcotics, Opioids, Morphine, Codeine,Oxycodone, Hydrocodone, Buprenorphine, Tramadol, Non-Narcotics,Paracetamol, acetaminophen, NSAIDS, and Flupirtine.

Examples of anesthetics include local anesthetics (e.g., Lidocaine,Benzocaine, and Ropivacaine) and general anesthetic.

Examples of tissue matrix degradation inhibitors that inhibit the actionof metalloproteinases (MMPs) and other proteases include MMP inhibitors(e.g., exogenous MMP inhibitors, hydroxamate-based MMP inhibitors,Batimastat (BB-94), Ilomastat (GM6001), Marimastat (BB2516), Thiols,Periostat (Doxycycline), Squaric Acid, BB-1101, Hydroxyureas,Hydrazines, Endogenous, Carbamoylphosphates, Beta Lactams, and tissueInhibitors of MMPs (TIMPs)).

Examples of anti-cancer agents include monoclonial antibodies,bevacizumab (Avastin), cellular/chemoattractants, alkylating agents(e.g., Bifunctional, Cyclophosphamide, Mechlorethamine, Chlorambucil,Melphalan, Monofunctional, Nitrosoureas and Temozolomide),anthracyclines (e.g., Daunorubicin, Doxorubicin, Epirubicin, Idarubicin,Mitoxantrone, and Valrubicin), cytoskeletal disrupters (e.g., Paclitaxeland Docetaxel), epothilone agents that limit cell division by inhibitingmicrotubule function, inhibitor agents that block various enzymes neededfor cell division or certain cell functions, histone deacetylaseinhibitors (e.g., Vorinostat and Romidepsin), topoisomerase I inhibitors(e.g., Irinotecan and Topotecan), topoisomerase II inhibitors (e.g.,Etoposide, Teniposide, and Tafluposide), kinase inhibitors (e.g.,Bortezomib, Erlotinib, Gefitinib, Imatinib, Vemurafenib, andVismodegib), nucleotide analogs (e.g., Azacitidine, Azathioprine,Capecitabine, Cytarabine, Doxifluridine, Fluorouracil, 5-FU, Adrucil,Carac, Efudix, Efudex, Fluoroplex, Gemcitabine, Hydroxyurea,Mercaptopurine, and Tioguanine), peptide antibiotic agents that cleaveDNA and disrupt DNA unwinding/winding (e.g., Bleomycin and Actinomycin),platinum-based anti-neoplastic agents that cross link DNA which inhibitsDNA repair and/or synthesis (e.g., Carboplatin, Cisplatin, Oxaliplatin,and Eloxatin), retinoids (e.g., Tretinoin, Alitretinoin, andBexarotene), vinca alkaloids gents that inhibit mitosis and microtubuleformation (e.g., Vinblastine, Vincristine, Vindesine, Vinorelbine),anti-ileus agents, pro-motility agents, immunosuppresants (e.g.,Tacrolimus), blood aspect modifier agents (e.g., Vasodilator, Viagra,and Nifedipine), 3-hydroxy-3-methyl-glutaryl-CoA (HMG CoA) reductaseinhibitors (e.g., Atorvastatin), and anti-angiogenesis agents.

Exemplary medicants also include agents that passively contribute towound healing such as, for example, nutrients, oxygen expelling agents,amino acids, collageno synthetic agents, Glutamine, Insulin, Butyrate,and Dextran. Exemplary medicants also include anti-adhesion agents,examples of which include Hyaluronic acid/Carboxymethyl cellulose(seprafilm), Oxidized Regenerated Cellulose (Interceed), and Icodextrin4% (Extraneal, Adept).

FIG. 24 and FIG. 25 illustrate one embodiment of an adjunct 1420 havinga medicant 1422 releasably retained therein. In this example, theadjunct 1420 is in the form of a sheet-like fiber woven mesh. As shownin FIG. 24 , the tight fibers of the adjunct 1420 in its originalconfiguration allow the medicant 1422 to be retained therein. When theadjunct 1420 is delivered at the treatment site, water and/or otheragents, shown schematically as drops 1424 a, 1424 b in FIG. 24 , areconfigured to cause the fibers to swell and elongate such that thedistances between the fibers increase, as shown in FIG. 25 . In thisway, the medicant 1422 is released, as also shown in FIG. 25 . A personskilled in the art will appreciate that the adjunct 1420 can be formedfrom different types of fibers. The fibers can have different absorptionrates, density, direction, patterns, size, and other properties that areselected so as to provide desired tissue re-growth. While some regionsof the adjunct can be configured to release at least one medicant so asto encourage tissue re-growth, one or more regions of the adjunct can beconfigured to release at least one medicant so as to discourage tissuere-growth.

FIG. 26 illustrates another embodiment of an adjunct 1426 in the form ofa laminate including heterogeneous portions or layers having differentdegradation rates and incorporating different medicants. As shown, theadjunct 1426 includes a top layer or portion 1428 and a bottom layer orportion 1430 that have different degradation rates. Furthermore, each ofthe top and bottom portions 1428, 1430 can have various portions havingdegradation rates that vary in a distinct or continuous manner Thedegradation rates can vary across the adjunct in a number of suitableways that depend on a desired treatment effect to be provided by theadjunct. In some embodiments, an adjunct can have a single degradationrate instead of having different degradation rates.

In the embodiment of FIG. 26 , the top portion 1428 of the adjunct 1426includes two portions 1428 a, 1428 b having different degradation rates.The bottom portion 1430 includes two portions 1430 a, 1430 b havingdifferent degradation rates. Each of the portions can include adifferent medicant such that, as a portion degrades, a respectivemedicant is eluted or released. The degradation rates and distributionof the medicants within one or more of the portions 1428 a, 1428 b, 1430a, 1430 b can further vary in a distinct or continuous manner such thatthe adjunct 1426 can provide an elution profile shown in a graph 1432 inFIG. 26 . As shown, a central area 1434 of the adjunct 1426 centeredaround a mid-portion 1436 thereof has an increased elution rate of oneor more medicants that peaks at the mid-portion 1436, whereas smalleramount of the medicant(s) is eluted from opposite sides of the adjunct1426 along its length 1426L. The increased elution rate can be due toproperties of the adjunct 1426 at the central area 1434 and theconcentration of the medicants.

As also shown in FIG. 26 , the adjunct 1426 is configured to releasemedicants in different elution profiles along the length 1426L thereofand along a width 1426W thereof. For example, the medicants can bereleased along the width 1426W as a bolus dose and along the length as atime-release dose. Release of one or more of the medicants can regulaterelease of at least one other of the medicants. However, the medicantscan be released in any other manner, depending on a desired treatment tobe delivered.

The adjunct 1426 has a generally rectangular shape, which may facilitateits use with a linear stapler. Other adjuncts can have a differentshape, such as to facilitate use thereof with a circular stapler. FIG.27 illustrates such an implementation of an adjunct 1436 configured foruse with a circular surgical stapler. The adjunct 1436 thus has agenerally circular shape.

The adjunct 1436 in the illustrated implementation of FIG. 27 is formedfrom a plurality of fibers and includes a plurality of heterogeneousfiber lattice sections 1438 a, 1438 b, 1438 c, 1438 d. The first fiberlattice section 1438 a is located on a top side and on an exterior sideof the adjunct 1436 and is configured to discourage tissue growth byhaving a first medicant (not shown) releasably retained therein that isconfigured to discourage tissue growth, such as an anti-adhesion agent.The second fiber lattice section 1438 b is located on a bottom side ofthe adjunct 1436 and is configured to encourage tissue growth by havinga second medicant (not shown) releasably retained therein that isconfigured to encourage tissue growth, such as a growth factor. Thethird fiber lattice section 1438 c is located on an interior side of theadjunct 1436 and is configured to facilitate hemostasis by having athird medicant (not shown) releasably retained therein that isconfigured to facilitate hemostasis, such as a hemostatic agent. Thefourth fiber lattice section 1438 d is located in an interior area ofthe adjunct 1436 and is configured to space apart the top and bottomsides of the adjunct 1436 to thereby space apart the tissuegrowth-encouraging and tissue growth-discouraging portions of theadjunct 1436. The fourth fiber lattice section 1438 d can have a fourthmedicant (not shown) releasably retained therein. The fourth medicantcan include, for example, an anti-adhesion agent or can include ORCand/or another hemostatic agent.

An adjunct can be implanted in a variety of ways. For example, theadjunct can be delivered using a surgical stapler introducedlaparoscopically.

Various embodiments of adjuncts, implanting adjuncts, and/or surgicalstaplers are discussed further in U.S. Pat. Pub. No. 2018/0353174 filedJun. 13, 2017 and entitled “Surgical Stapler with Controlled Healing,”U.S. Pat. No. 10,569,071 entitled “Medicant Eluting Adjuncts And MethodsOf Using Medicant Eluting Adjuncts” issued Feb. 25, 2020, U.S. Pat. No.10,716,564 entitled “Stapling Adjunct Attachment” issued Jul. 21, 2020,U.S. Pat. Pub. No. 2013/0256377 entitled “Layer Comprising DeployableAttachment Members” filed Feb. 8, 2013, U.S. Pat. No. 8,393,514 entitled“Selectively Orientable Implantable Fastener Cartridge” filed Sep. 30,2010, U.S. Pat. No. 8,317,070 entitled “Surgical Stapling Devices ThatProduce Formed Staples Having Different Lengths” filed Feb. 28, 2007,U.S. Pat. No. 7,143,925 entitled “Surgical Instrument Incorporating EAPBlocking Lockout Mechanism” filed Jun. 21, 2005, U.S. Pat. Pub. No.2015/0134077 entitled “Sealing Materials For Use In Surgical Stapling”filed Nov. 8, 2013, U.S. Pat. Pub. No. 2015/0134076, entitled “HybridAdjunct Materials for Use in Surgical Stapling” filed on Nov. 8, 2013,U.S. Pat. Pub. No. 2015/0133996 entitled “Positively Charged ImplantableMaterials and Method of Forming the Same” filed on Nov. 8, 2013, U.S.Pat. Pub. No. 2015/0129634 entitled “Tissue Ingrowth Materials andMethod of Using the Same” filed on Nov. 8, 2013, U.S. Pat. Pub. No.2015/0133995 entitled “Hybrid Adjunct Materials for Use in SurgicalStapling” filed on Nov. 8, 2013, U.S. Pat. Pub. No. 2015/0272575entitled “Surgical Instrument Comprising a Sensor System” and filed onMar. 26, 2014, U.S. Pat. Pub. No. 2015/0351758 entitled “AdjunctMaterials and Methods of Using Same in Surgical Methods for TissueSealing” filed on Jun. 10, 2014, U.S. Pat. Pub. No. 2013/0146643entitled “Adhesive Film Laminate” filed Feb. 8, 2013, U.S. Pat. No.7,601,118 entitled “Minimally Invasive Medical Implant And InsertionDevice And Method For Using The Same” filed Sep. 12, 2007, and U.S. Pat.Pub. No. 2013/0221065 entitled “Fastener Cartridge Comprising AReleasably Attached Tissue Thickness Compensator” filed Feb. 8, 2013,which are each hereby incorporated by reference herein in theirentireties.

In an exemplary embodiment, the adjunct is bioabsorbable andbiocompatible. In such embodiments, the material(s) forming the adjunctcan include bioabsorbable and biocompatible polymers, includinghomopolymers and copolymers. Examples of homopolymers and copolymersinclude p-dioxanone (PDO or PDS), polyglycolic acid (PGA),poly(lactic-co-glycolic acid) (PLGA), polycaprolactone (PCL),trimethylene carbonate (TMC), and polylactic acid (PLA), poly(glycolicacid-co-lactic acid) (PLA/PGA) (e.g., PLA/PGA materials used in Vicryl®,Vicryl Rapide™, PolySorb, and Biofix), polyurethanes (such as Elastane,Biospan, Tecoflex, Bionate, and Pellethane fibers), polyorthoesters,polyanhydrides (e.g., Gliadel and Biodel polymers), polyoxaesters,polyesteramides, and tyrosine-based polyesteramides. The copolymers canalso include poly(lactic acid-co-polycaprolactone) (PLA/PCL),poly(L-lactic acid-co-polycaprolactone) (PLLA/PCL), poly(glycolicacid-co-trimethylene carbonate) (PGA/TMC) (e.g., Maxon), Poly(glycolicacid-co-caprolactone) (PCL/PGA) (e.g., Monocryl and Capgly), PDS/PGA/TMC(e.g., Biosyn), PDS/PLA, PGA/PCL/TMC/PLA (e.g., Caprosyn), andLPLA/DLPLA (e.g., Optima).

A location of where to implant the adjunct at the duodenum at a locationthat corresponds to where a mucosal ablation location inside theduodenum can be determined, for example, using a fiducial marker that ispositioned inside the duodenum. The fiducial marker can be magnetic,thereby allowing the fiducial marker inside the duodenum to be locatedmagnetically from outside the duodenum without naked eye or visiblelight visualization of the fiducial marker or the balloon to which thefiducial marker is attached. Various embodiments of using a fiducialmarker to determine a location of an ablation device and/or a scopethrough which an ablation device has been advanced are discussed furtherbelow.

For example, in a DMR procedure in which an endoscope and a laparoscopeare used to visualize inside and outside a duodenum, an implantable,nerve-stimulating sleeve or stent can be implanted at the duodenumaround an outer diameter of the duodenum. The implanted sleeve or stentis configured to supply an electrical current to the duodenum toconstrict blood supply and length effect of the ablation. The electricalcurrent is configured to stimulate a nerve, such as a vague nerve, andthereby limit the sensing in the patient's gastrointestinal tract at theduodenum and thereby further prevent signal transmission of the sensingto another part of the patient's body to improve therapeutic effect ofthe DMR procedure. The nerve stimulation can be configured to limit thesensing in the patient's gastrointestinal tract and enhance the effectof mucosal ablation by overwhelming nerve signals or stimulating themout of sequence from eating. The mucosal ablation and the nervestimulation may thus each contribute to the procedure's therapeuticeffect.

The electrical current applied to an exterior of the duodenum may allowfor a lower current to be used for ablation and/or for the ablationdevice to be more precisely positioned to ablate specific areas of themucosa that are not subject to the stimulation. The electrical currentcan be delivered using an electrode attached to the sleeve or stent, forexample.

The implanted sleeve or stent can be configured to deliver theelectrical stimulation in response to a trigger event, such as detectionof the patient eating.

The sleeve or stent can be configured to constrict around the entirecircumference of the duodenum or only at specific areas around thecircumference of the duodenum. The sleeve or stent constricting only atspecific areas around the circumference of duodenum may minimize anyconstriction effect of the sleeve or stent around the duodenum, as theduodenum should not be squeezed shut or otherwise overly reduced indiameter to hinder normal intestinal function.

A location of where to implant the sleeve or stent outside the duodenumat a location that corresponds to a mucosal ablation location inside theduodenum can be determined, for example, using a fiducial marker that ispositioned inside the duodenum. The fiducial marker can be magnetic,thereby allowing the fiducial marker inside the duodenum to be locatedmagnetically from outside the duodenum without naked eye or visiblelight visualization of the fiducial marker or the element to which thefiducial marker is attached. Various embodiments of using a fiducialmarker to determine a location of an ablation device and/or a scopethrough which an ablation device has been advanced are discussed furtherbelow.

The sleeve or stent can be bioabsorbable and biocompatible. In suchembodiments, the material(s) forming the sleeve or stent can includebioabsorbable and biocompatible polymers, including homopolymers andcopolymers.

Various embodiments of nerve stimulation, sensing and reacting to foodingestion, and implants configured to provide electrical stimulation ofnerves are further described in U.S. Pat. No. 5,188,104 issued Feb. 23,1993 and entitled “Treatment Of Eating Disorders By Nerve Stimulation,”U.S. Pat. No. 5,231,988 issued Aug. 3, 1993 and entitled “Treatment OfEndocrine Disorders By Nerve Stimulation,” U.S. Pat. No. 5,263,480issued Nov. 23, 1993 and entitled “Treatment Of Eating Disorders ByNerve Stimulation,” U.S. Pat. No. 5,540,730 issued Jul. 30, 1996 andentitled “Treatment Of Motility Disorders By Nerve Stimulation,” U.S.Pat. No. 8,352,026 issued Jan. 8, 2013 and entitled “Implantable PulseGenerators And Methods For Selective Nerve Stimulation,” U.S. Pat. No.9,044,606 issued Jun. 2, 2015 and entitled “Methods And Devices ForActivating Brown Adipose Tissue Using Electrical Energy,” U.S. Pat. No.10,092,738 issued Oct. 9, 2018 and entitled “Methods And Devices ForInhibiting Nerves When Activating Brown Adipose Tissue,” U.S. Pat. Pub.No. 2009/0132018 filed Nov. 16, 2007 and entitled “Nerve StimulationPatches And Methods For Stimulating Selected Nerves,” U.S. Pat. Pub. No.2008/0147146 filed Dec. 19, 2006 and entitled “Electrode Patch AndMethod For Neurostimulation,” U.S. Pat. Pub. No. 2005/0277998 filed Jun.7, 2005 and entitled “System And Method For Nerve Stimulation,” U.S.Pat. Pub. No. 2006/0195153 filed Jan. 31, 2006 and entitled “System AndMethod For Selectively Stimulating Different Body Parts,” U.S. Pat. Pub.No. 2007/0185541 filed Aug. 2, 2006 and entitled “Conductive Mesh ForNeurostimulation,” U.S. Pat. Pub. No. 2006/0195146 filed Jan. 31, 2006and entitled “System And Method For Selectively Stimulating DifferentBody Parts,” U.S. Pat. Pub. No. 2008/0132962 filed Dec. 1, 2006 andentitled “System And Method For Affecting Gastric Functions,” U.S. Pat.Pub. No. 2008/0147146 filed Dec. 19, 2006 and entitled “Electrode PatchAnd Method For Neurostimulation,” U.S. Pat. Pub. No. 2009/0157149 filedDec. 14, 2007 and entitled “Dermatome Stimulation Devices And Methods,”U.S. Pat. Pub. No. 2009/0149918 filed Dec. 6, 2007 and entitled“Implantable Antenna,” U.S. Pat. Pub. No. 2009/0132018 filed Nov. 16,2007 and entitled “Nerve Stimulation Patches And Methods For StimulatingSelected Nerves,” U.S. Pat. Pub. No. 2010/0161001 filed Dec. 19, 2008and entitled “Optimizing The Stimulus Current In A Surface BasedStimulation Device,” U.S. Pat. Pub. No. 2010/0161005 filed Dec. 19, 2008and entitled “Optimizing Stimulation Therapy Of An External StimulatingDevice Based On Firing Of Action Potential In Target Nerve,” U.S. Pat.Pub. No. 2010/0239648 filed Mar. 20, 2009 and entitled “Self-Locating,Multiple Application, And Multiple Location Medical Patch Systems AndMethods Therefor,” U.S. Pat. Pub. No. 2011/0094773 filed Oct. 26, 2009and entitled “Offset Electrode,” and U.S. Pat. No. 8,812,100 filed May10, 2012 and entitled “A Device And Method For Self-Positioning Of AStimulation Device To Activate Brown Adipose Tissue Depot InSupraclavicular Fossa Region,” which are hereby each incorporated byreference in their entireties.

For yet another example, in a DMR procedure in which an endoscope and alaparoscope are used to visualize inside and outside a duodenum, animplantable suture can be implanted at the duodenum by being wrappedaround an outer diameter of the duodenum. A location of where to wrapthe suture outside the duodenum at a location that corresponds to amucosal ablation location inside the duodenum can be determined, forexample, using a fiducial marker on the ablation device, such as on aballoon of the ablation device or on another element thereof, that ispositioned inside the duodenum. Various embodiments of using a fiducialmarker to determine a location of an ablation device and/or a scopethrough which an ablation device has been advanced are discussed furtherbelow.

The suture is a medicant-eluting suture and/or an antimicrobial suture,which allows the suture to provide treatment to the duodenum fromoutside the duodenum to help the duodenum heal properly after theablation. The medicant eluted by the medicant-eluting suture can beconfigured to limit sensing in the gastrointestinal tract at theduodenum and thereby prevent signal transmission of the sensing toanother part of the patient's body. The mucosal ablation and the suturemay thus each contribute to the DMR procedure's therapeutic effect.

The suture can be spiral-shaped so as to wrap helically around theduodenum. The spiral shape may minimize any constriction effect of thesuture around the duodenum, as the duodenum should not be tied shut orotherwise overly reduced in diameter to hinder normal intestinalfunction.

In an exemplary embodiment, the suture is bioabsorbable andbiocompatible. In such embodiments, the material(s) forming the suturecan include bioabsorbable and biocompatible polymers, includinghomopolymers and copolymers.

Controlling Intelligent Surgical Instruments

Devices, systems, and methods for multi-source imaging provided hereinmay allow for controlling intelligent surgical instruments. An imagingsystem can be configured to visualize a surgical site during performanceof a surgical procedure, as discussed herein. As also discussed herein,a surgical device such as an intelligent surgical instrument can be usedin performing the surgical procedure. The surgical device can be in useat the surgical site while the imaging system is providingvisualization, but the imaging system's view of the surgical device maybe obstructed such that images gathered by the imaging system do notshow the surgical device fully or at all.

The obstructed view can be caused, for example, by a tissue blocking theimaging system's view of the surgical device, such as if the imagingsystem is positioned on a first side of a tissue wall and the surgicaldevice is positioned on a second, opposite side of the tissue wall. ADMR procedure is one example of a surgical procedure in which a surgicaldevice can be positioned in a duodenum so as to be positioned on a firstside of a tissue wall defined by the duodenum and an imaging device canbe positioned outside the duodenum so as to be positioned on a second,opposite side of the tissue wall defined by the duodenum. A lungresection is another example of a surgical procedure in which a surgicaldevice can be positioned in a lung so as to be positioned on a firstside of a tissue wall defined by the lung and an imaging device can bepositioned outside the lung so as to be positioned on a second, oppositeside of the tissue wall defined by the lung. A colectomy is anotherexample of a surgical procedure in which a surgical device can bepositioned in a colon so as to be positioned on a first side of a tissuewall defined by the colon and an imaging device can be positionedoutside the colon so as to be positioned on a second, opposite side ofthe tissue wall defined by the colon. EMR and ESD are other examples ofa surgical procedure in which a surgical device can be positioned in astomach so as to be positioned on a first side of a tissue wall definedby the stomach and an imaging device can be positioned outside thestomach so as to be positioned on a second, opposite side of the tissuewall defined by the stomach. Other surgical procedures can be performedin which a surgical device is positioned on a first side of a tissuewall and an imaging device can be positioned on a second, opposite sideof the tissue wall.

A tissue can block the imaging system's view of the surgical devicewithout the imaging system and the surgical device being on opposedsides of a tissue wall, such as if a tissue shifts position duringperformance of the surgical procedure and thus obstructs a view of thesurgical device that the imaging system had before the tissue shift.

Regardless of the cause of the imaging system's obstructed view of thesurgical device, the imaging system having an obstructed view of thesurgical device may make control of the surgical device more difficult.A medical practitioner viewing images gathered by the imaging device andcontrolling the surgical device may not be able to make fully informeddecisions about controlling the surgical device since the view of thesurgical device is obstructed and may prevent the medical practitionerfrom seeing information that would otherwise factor into control of thesurgical device. A controller at a surgical hub, a robotic surgicalsystem, or other computer system controlling the surgical device may notbe able to make fully informed decisions about controlling the surgicaldevice since the view of the surgical device is obstructed and mayprevent the controller from detecting information in the images thatwould otherwise factor into the controller's control of the surgicaldevice.

An imaging device that has an obstructed view of an intelligent surgicaldevice at a surgical site can be configured to visualize the surgicalsite and thereby monitor a parameter of a tissue engaged by the surgicaldevice, such as by the surgical device ablating the tissue, grasping thetissue, stapling the tissue, or otherwise engaging the tissue. Acontroller, such as a controller at a surgical hub, a robotic surgicalsystem, or other computer system, in communication with the imagingdevice and the surgical device can receive a signal from the imagingdevice regarding the monitored parameter. The controller can receive thesignal directly from the imaging device or through one or moreintermediary devices. As discussed above, an algorithm stored on boardthe intelligent surgical device or stored elsewhere can include one ormore variable parameters. The controller can be configured to adjust atleast one variable parameter of the algorithm based on the monitoredparameter, as indicated by the received signal. The surgical device canthus be controlled based on information gathered by the imaging devicedespite the imaging device having an obstructed view of the surgicaldevice.

The imaging device is configured to gather images, as discussed herein.The gathering of the images can be how the imaging device monitors theparameter such that the imaging device's normal operation can allow theparameter to be monitored. For example, as discussed above, an imagingdevice can be configured to gather images using invisible light. Theinvisible light can allow the imaging device to gather images on anopposite side of a tissue wall from where the imaging device ispositioned because invisible light can “see” through the tissue wall.Thus, the imaging device being configured to gather images usinginvisible light can allow the imaging device to monitor the parameter.

As mentioned above, examples of variable parameters of a surgicaldevice's algorithm include motor speed, motor torque, energy level,energy application duration, tissue compression rate, jaw closure rate,cutting element speed, load threshold, and other parameters. In anexemplary embodiment, the variable parameter(s) changed based on themonitored parameter can affect a movement of the surgical device, anelectrode of the surgical device that is configured to deliver energy tothe tissue (in embodiments in which the surgical device including anelectrode, such as with an ablation device), a power level of thesurgical device, or voltage control of the surgical device.

FIG. 28 illustrates one embodiment of an intelligent ablation device(ablation probe) 1440 positioned in a lung to apply energy to a tumor1442 in the lung. An imaging device 1444 is positioned outside the lungand is configured to gather images using at least infrared light, e.g.,by using an infrared (IR) camera. The ablation device 1440 is thuspositioned on a first side of a tissue wall 1446 defined by the lung,and the imaging device 1444 is positioned on a second, opposite side ofthe tissue wall 1446 so as to have an obstructed view of the ablationdevice 1440. The imaging device's IR capability, however, lets theimaging device 1444 “see” inside the lung and gather internal lungtemperature information. Infrared images 1448 gathered by the IR thermalcamera at four times t₀, t₂, t₄, t₆ are shown in FIG. 28 .

FIG. 28 also shows a graph indicating each of IR camera temperature (°C.), ablation probe 1440 position (cm), and ablation probe 1440 powerlevel (W) versus time. As indicated in the graph, the IR thermal cameramonitors the temperature of an area including the ablation device'sablation zone (e.g., an area of ablation that an ablation device 1440can create), which includes the tumor 1442 and a margin area around thetumor 1442 within the ablation zone. The times in the graph include thefour times t₀, t₂, t₄, t₆ for which IR images are shown.

The therapeutic temperature range for tissue ablation is in a range fromabout 60° C. to about 100° C. As shown in the graph, the power level iscontrolled based on the monitored temperature, as indicated in thegathered IR images, so the tumor 1442 is being ablated within thetherapeutic temperature range from a time between times t₁ and t₂ towhen energy application stops shortly before time t₆ between times t₅and t₆. The temperature decreases from time t₂ and t₃, so at least onevariable parameter of the ablation device's algorithm, such as theablation device's power level and/or other variable parameter thataffects energy delivered by an electrode of the ablation device 1440, ischanged at time t₃ to increase power level and thus prevent thetemperature from falling below about 60° C. The temperature increasesfrom time t₃ and t₄, so the variable parameter(s) are changed again attime t₄ to reduce power level and thus prevent the temperature fromrising above about 100° C. The temperature decreases again shortlybefore time t₅, so the variable parameter(s) are changed again at timet₅ to increase power level and thus prevent the temperature from fallingbelow about 60° C. As discussed above, a controller at a surgical hub, arobotic surgical system, or other computer system can change thevariable parameter(s) and can control execution of the algorithm.

Tissue being at a temperature up to about 41° C. can cause blood vesseldilation and increased blood perfusion and trigger a heat-shock responsebut have little long term effect. Tissue being at a temperature aboveabout 41° C. makes the tissue susceptible to or causes the tissue toincur irreversible cell damage, where the higher the temperature, thegreater the damage. As shown in the graph and as discussed furtherbelow, variable parameter(s) are also adjusted in this illustratedembodiment to keep the temperature of healthy, non-targeted lung tissuebelow about 41° C. while maintaining effective ablation of the tumor1442.

FIG. 28 is described with respect to a lung, but similar control can beperformed with respect to a surgical procedure performed at anotherhollow organ or body lumen. For example, in a DMR procedure, anintelligent surgical device, such as an intelligent ablation device, canbe positioned inside a duodenum and an imaging device, such as alaparoscope, can be positioned outside the duodenum. The imaging devicewill thus have an obstructed view of the ablation device because of theduodenum's intervening tissue wall. The imaging device can be configuredto gather images indicating temperature information, such as by imagingusing at least infrared light, e.g., by using an IR camera, similar tothat discussed above regarding the imaging device 1444 of FIG. 28 . Theintelligent ablation device can thus be controlled similar to thatdiscussed above regarding the ablation device 1440 of FIG. 28 (and asdiscussed further below).

In some embodiments, an intelligent surgical device (e.g., anintelligent ablation device or other device) can include an electrodeconfigured to apply energy to tissue, and tissue contact integrity canbe sensed to control the surgical device's power level (e.g., bychanging at least one variable parameter) in addition to another aspectof the surgical device such as power change tissue impedance thresholdor other parameter controllable via one or more variable parameters.Such sensing may prevent electrode-tissue contact to less than ananticipated amount of contact given the electrode's known size andshape, may prevent a cross-sectional area of the electrode resulting inan inadvertent level of power density overly concentrating a cauterylevel being provided by the electrode, may prevent RF output voltageshigher than necessary based on a tissue parameter such an impedance,and/or may reduce arcing potential.

An ablation device, for example the ablation device 1410 of FIG. 23 andother embodiments of ablation devices described herein, can be abi-polar device that includes an inflatable or expandable member such asa balloon or basket and a plurality of electrodes (an electrode array)attached to the inflatable or expandable member that are configured tocontact tissue at least when the inflatable or expandable member isinflated or expanded in a hollow organ or body lumen. A return can beprovided with segmented electrodes contacting and applying pressure toan outer diameter of the hollow organ or body lumen. Various embodimentsof return electrodes are discussed further below. Applying pressure tothe outer diameter may help control a gap between the tissue intended tobe ablated and adjacent tissue not intended to be ablated. Impedancemeasurements can be sampled until the return pressure/electrode contactarea creates a desired impedance band. For example, in a DMR procedure,duodenal mucosa tissue to be ablated is lifted, such as with saline, tohelp protect the duodenum's outer layers, as discussed herein. Applyingpressure to the duodenum's outer diameter may thus help control the gapbetween the target duodenal mucosa tissue intended to be ablated and thenon-targeted duodenum outer layers not intended to be ablated.Additional saline can be added to adjust the gap as necessary.Additional saline also adjusts the tissue impedance. A pressure of thetissue can be sensed, for example, using a strain gauge, which may helpadjustment of the applied outer pressure and/or of an amount and/ordelivery rate of the saline.

The segmented electrodes can be a single return or an arc of returns,such as a 90° arc of returns. Various embodiments of return electrodesare discussed further below. A sizing tool can be used around acircumference of the hollow organ or body lumen to determine propersizing and/or spacing for the return, which may allow for a full 360°ablation with a single ablation cycle.

FIG. 29 and FIG. 30 illustrate one embodiment of a flexible force probe1450 configured to apply force to a tissue wall 1452 from one side ofthe tissue wall 1452, which in this illustrated embodiment is outside ofa body lumen 1454 in which a scope 1456 is positioned. The tissue lumen1454 is not shown in FIG. 30 . The flexible force probe 1450 includes anarm 1450 a configured to serve as a return and to abut and press againstthe tissue wall 1452. The arm 1450 a being flexible may allow the arm1450 a to conform to the shape of the tissue wall 1452. The internalpath of the scope 1456 can be known within the lumen 1454, so the probe1450 external to the lumen 1454 can follow the same spline path with thearm 1450 a pressing on the tissue wall 1452. The arms 1450 a can thus bein position to press against the exterior surface of the lumen 1454where an ablation device introduced into the lumen 1454 ablates thelumen 1454 from inside the lumen 1454.

In some embodiments, a ground pad can be applied to an exterior oftissue being ablated, e.g., an exterior surface of a duodenum or othertissue. The ground pad is configured to provide a current path to helpcontain ablation to the ablation zone. Ablation zone size is known for aparticular ablation device. For example, NeuWave™ ablation probes(available from Ethicon US LLC of Cincinnati, Ohio) have an ablationzone of 2 cm. The ground pad can extend a length broader than anablation device's ablation zone to help ensure that the entire ablationzone receives the benefit of the ground pad. A location of the ablationzone (e.g., location of the electrode providing the ablation zone) canbe determined from outside the tissue being ablated from within by, forexample, using multi-spectral imaging or using electromagnetic or RFmonitoring, such as by using a laparoscope positioned outside a duodenumthat is being ablated with an ablation device inside the duodenum.Various embodiments of determining a location of an ablation deviceand/or a scope in a hollow organ or body lumen are discussed furtherbelow.

In some embodiments, an intelligent surgical device (e.g., anintelligent ablation device or other device) can include an electrodeconfigured to apply energy to tissue, and the surgical device's controlalgorithm can be adjusted (e.g., at least one variable parameter of thealgorithm changed) to calibrate control of the electrode based on atleast one measured tissue parameter before energy application begins.Energy may thus be more efficiently applied from a start of the ablationand/or the ablation may be completed more quickly and thus reduce chanceof damaging any nearby tissue not intended to be ablated (non-targetedtissue).

In an exemplary embodiment, the tissue parameter used in calibratingcontrol of the electrode includes at least one of tissue temperature,tissue impedance, and tissue thickness. As discussed herein, varioussurgical procedures can involve an imaging device being positionedoutside a hollow organ or body lumen and a surgical device beingpositioned inside the hollow organ or body lumen such that the imagingdevice has an obstructed view of the surgical device. The imaging devicecan be configured to measure an exterior temperature of the hollow organor body lumen, such as an external surface temperature being measuredvia one or more imaging modalities, and the surgical device can beconfigured to measure an interior temperature of the hollow organ orbody lumen, such as an internal surface temperature being measured usinga temperature sensor measuring. In many instances, the exterior andinterior temperatures will not match due to, e.g., the tissue'sthickness. In general, locations with a greater wall thickness will becooler at the external surface than locations of the same tissue withless thickness. Additionally, thickness of a same tissue wall willusually vary between patients and can also vary for a tissue wall in aparticular patient depending on where axially along the tissue and wherecircumferentially around the tissue that the thickness is measured.

FIG. 31 , FIG. 32 , and FIG. 33 illustrate one embodiment of using atleast one measured tissue parameter in calibrating control of anelectrode. In this illustrated embodiment, first and second electrodes1460, 1462 are each positioned to contact an interior surface 1464 of abody lumen 1466. The body lumen 1466 can be, for example, a duodenumbeing ablated in a DMR procedure using an ablation device that includesthe first and second electrodes 1460, 1462 attached to an inflatable orexpandable member such as a balloon or a basket (see for example FIG. 23and other embodiments of ablation devices described herein).

Each of the first and second electrodes 1460, 1462 is configured tomonitor a temperature T1, T2 of the body's lumen's interior surface(also referred to herein as “internal surface”) 1464, such as by each ofthe first and second electrodes 1460, 1462 including an integratedtemperature sensor configured to monitor temperature or by each of thefirst and second electrodes 1460, 1462 including an integrated IR sensorconfigured to measure IR and emit an IR frequency signal thatcorresponds to a specific temperature. The temperature of the body'slumen's interior surface 1464 can thus be monitored at first and secondlocations 1468, 1470 around an interior circumference of the body lumen1466 that correspond to the locations of the first and second electrodes1460, 1462. A different number of electrodes can be used in otherembodiments, with a corresponding different number of internal surfacetemperature measurements being gathered.

A temperature of an exterior surface (also referred to herein as“external surface”) 1472 of the body lumen 1466 is also measured in thisillustrated embodiment. The exterior surface 1472 temperature T1′, T2′is monitored at first and second locations 1474, 1476 around an exteriorcircumference of the body lumen 1466 that correspond to the first andsecond locations 1468, 1470 around the interior circumference of thebody lumen 1466 and thus to the locations of the first and secondelectrodes 1460, 1462. The external surface 1472 temperatures T1′, T2′can be measured, for example, by using thermal imaging provided by animaging device (not shown), such as a laparoscope, that is positionedoutside of the body lumen 1466, by using a temperature sensor (e.g., atemperature sensor on a flexible force probe or other surgical deviceadvanced through a working channel of the imaging device), or by usingan IR sensor (e.g., an IR sensor on a flexible force probe or othersurgical device advanced through a working channel of the imagingdevice). A different number of electrodes can be used in otherembodiments, with a corresponding different number of external surfacetemperature measurements being gathered.

An increase in temperature at an exterior of a hollow organ or bodylumen will be proportional to progress of denaturation in the underlyingtissue wall. Measuring exterior tissue temperature may thus be tied tothe ablation occurring underneath the location where exterior tissuetemperature was measured. Accordingly, measuring internal and externaltemperatures T1, T2, T1′, T2′ of the body lumen 1466 allows atemperature gradient to be established from outside the serosal layer tothe mucosal layer inside the lumen 1466 such that a temperature of eachtissue layer can be established. The internal and external temperaturesT1, T2, T1′, T2′ will typically not be the same before ablation beginsbecause the mucosal layer acts as an insulator. The internal andexternal temperatures T1, T2, T1′, T2′ will typically not be the sameduring ablation since the internal surface 1464 of the body lumen 1466is being heated. However, the exponential or polynomial relationshipbetween internal and external temperature is consistent for a sametissue wall thickness and tissue type. Calibration of the internal andexternal temperatures T1, T2, T1′, T2′ before ablation begins can definea relationship between the first internal and external temperatures T1,T1′ and the second internal and external temperatures T2, T2′. Measuringinternal and external temperatures at more than one location around ahollow organ or body lumen's circumference, as in this illustratedembodiment, may help account for different in tissue wall thicknessaround the circumference.

Measuring the external temperature T1′, T2′ can help ensure that theheating of the internal surface 1464 of the body lumen 1466 does notoverheat the tissue's outer layers that are unintended targets of theablation. As discussed above, tissue being at a temperature above about41° C. begins to make the tissue susceptible to or causes the tissue toincur irreversible cell damage. When the external temperature T1′, T2′is determined to be above a predetermined maximum external temperaturethreshold, such as 41° C., 50° C., 60° C., 70° C., or other temperature,at least one variable parameter of the ablation control algorithm can bechanged (e.g., by a controller of a surgical hub, a robotic surgicalsystem, or other computer system), such as reducing a power level of theablation device, to reduce heating of the tissue beyond the intendedinternal tissue. The predetermined maximum external temperaturethreshold being 60° C. or less may help prevent heating non-targetedtissue above 60° C., which as mentioned above is about the temperaturethat begins the therapeutic temperature range for tissue ablation. Thepredetermined maximum external temperature threshold being 50° C. orless can help prevent heating non-targeted tissue above 50° C., as about50° C. is when irreversible tissue damage begins to occur. Changing theat least one variable parameter can be such that the electrode(s)associated with the at least one variable parameter continue deliveringpower but at a different level, e.g., in an effort to reducetemperature, increase temperature, or maintain temperature as desired,or can be such that power is turned off such that the electrode(s)associated with the at least one variable parameter stop deliveringpower, e.g., because the target tissue being treated has been heated toa predetermined goal temperature.

Measuring the internal temperature T1, T2 can help ensure that theheating of the internal surface 1464 of the body lumen 1466 is withinthe therapeutic temperature range for tissue ablation. As discussedabove, the therapeutic temperature range for tissue ablation is in arange from about 60° C. to about 100° C. When the internal temperatureT1, T2 is determined to be above a predetermined maximum internaltemperature threshold, such as 100° C. or other temperature, or below apredetermined minimum internal temperature threshold, such as 60° C. orother temperature, at least one variable parameter of the ablationcontrol algorithm can be changed (e.g., by a controller of a surgicalhub, a robotic surgical system, or other computer system), such aschanging a power level of the ablation device, to stop heating or tomaintain effective heating of the tissue at the intended internaltissue.

The ablation device that includes the first and second electrodes 1460,1462 can include a fiducial marker, such as a magnetic fiducial marker,thereon, such as on a balloon of the ablation device or on anotherelement thereof. Various embodiments of fiducial markers are discussedfurther below. The external imaging device located outside the bodylumen 1466 can be configured to detect the fiducial marker to helpestimate tissue thickness 1478, 1480 at the first and second locations.The first thickness 1478 is greater than the second thickness 1480 inthis illustrated embodiment. The imaging device has a known location, sothe detected fiducial marker can allow a distance to be calculatedtherebetween that corresponds to the tissue thickness 1478, 1480. Tissuethickness is a variable that can be used to determine temperature.Tissue thickness is also a variable that can affect ablation devicepower output.

In embodiments in which infrared is used to measure internal andexternal temperatures T1, T2, T1′, T2′, each of the internal device(e.g., ablation device) and the external device (e.g., laparoscope) caninclude an IR emitter and receiver so that calibration can be achievedin both directions.

FIG. 31 and FIG. 32 illustrate a first tissue thickness 1478 between thefirst internal and external locations 1468, 1476 and a second tissuethickness 1480 between the second internal and external locations 1470,1478 at time point 1. FIG. 32 shows a portion of FIG. 31 , delineated byhash lines in FIG. 31 , that includes the first electrode 1460, thefirst internal location 1468, and the first external location 1474 ateach of four time points (time point 1, time point 2, time point 3, andtime point 4) during performance of a surgical procedure.

FIG. 33 shows a graph where the first internal and external temperaturesT1, T1′, estimated tissue thickness based on the first and secondestimated tissue thicknesses 1478, 1480, power level of the ablationdevice that includes the electrodes 1460, 1462, and tissue impedance areeach plotted versus time, including each of the four times points ofFIG. 32 . As shown in the graph, power begins being supplied to theelectrodes 1460, 1462 at time point 1 such that the electrodes 1460,1462 begin delivering energy to the tissue at time point 1. Energydelivery continues through time points 2 and 3 and ends at time point 4.As shown in FIG. 32 , the first tissue thickness 1478 decreases overtime during ablation. The second tissue thickness 1480 also decreasesduring ablation. FIG. 32 also shows that the first internal and externaltemperatures T1, T1′ increase over time during ablation even thoughpower level of the ablation device is decreasing over time duringablation, as heating can provide a cumulative effect. The secondinternal and external temperatures T2, T2′ also increase over timeduring ablation. FIG. 32 also shows that tissue impedance increases overtime during ablation, as the tissue becomes hotter and the tissue'sthickness decreases. At time point 4, ablation stops, e.g., power levelgoes to zero, in response to the first external temperature T1′ beingdetermined to be above the predetermined maximum external temperaturethreshold.

Controlling Pressure or Fluid Flow

Devices, systems, and methods for multi-source imaging provided hereinmay allow for controlling pressure or fluid flow. As discussed herein, asurgical procedure can include ablating tissue using an electrode. Asalso discussed herein, such a surgical procedure can include liftingtissue, such as by introduction of a fluid, before ablation to helpprotect non-targeted tissue from being overly heated. In an exemplaryembodiment, electrode pressure on tissue and/or fluid expulsion can becontrolled based on one of at least one monitored parameter of thetissue, a contact of the electrode with the tissue, and an aspect ofenergy transfer from the electrode to the tissue. Controlling a contactpressure of the electrode on tissue to which the electrode is deliveringenergy may help improve conductivity and/or may help direct energy ofthe electrode evenly and predictably. Controlling fluid expulsion, suchas expulsion of saline or other fluid used to lift tissue beforeablation, may help improve electrical coupling of the electrode with thetissue.

As discussed herein, an ablation device can be introduced into aduodenum or other anatomic structure through a working channel of ascope, e.g., an endoscope, and can expand radially outward once advanceddistally beyond the scope. FIG. 23 illustrates one embodiment of such anablation device 1410 that includes an electrode and an expandable orinflatable balloon 1412.

FIG. 34 , FIG. 35 , and FIG. 36 illustrate another embodiment of anablation device 1490 configured to expand radially outward and compressradially inward. The ablation device 1490 includes a plurality ofelectrodes that are configured to move between a compressedconfiguration, which facilitates movement of the ablation device 1490into and out of a patient, and an expanded configuration, whichfacilitates energy application to tissue by the electrodes that eachcontacts the tissue. The electrodes are spaced equally about a center ofthe device 1490 in this illustrated embodiment. FIG. 34 shows theablation device 1490 in a compressed configuration in which the ablationdevice's electrodes extend linearly. FIG. 35 and FIG. 36 show theablation device 1490 in an expanded configuration in which theelectrodes are radially expanded. FIG. 36 also shows the ablation device1490 advanced into position relative to a tumor 1492 through anendoscope 1494, with the electrodes expanded and advanced distally outof the ablation device's sheath 1496. The distal advancement of theelectrodes out of the sheath 1496 (or proximal movement of the sheath1496 relative to the electrodes) causes the electrodes to automaticallyradially expand. Correspondingly, proximal movement of the electrodesinto the sheath 1496 (or distal movement of the sheath 1496 relative tothe electrodes) causes the electrodes to automatically radiallycontract. In some embodiments, the sheath 1496 may be omitted such thatmovement of the electrodes into and out of the endoscope 1494 causesexpansion and compression of the electrodes.

FIG. 37 illustrates another embodiment of an ablation device 1500configured to expand radially outward. The ablation device 1500 includesa basket 1502 defined by a plurality of compressible strands or wires1504 that each have an electrode 1506 attached thereto. The basket 1502,and thus the electrodes 1506 attached thereto, is configured to movebetween a compressed configuration, which facilitates movement of theablation device 1500 into and out of a patient, and an expandedconfiguration, which facilitates energy application to tissue by theelectrodes 1506 that each contacts tissue. The ablation device 1500 alsoincludes a sheath 1508. Distal advancement of the basket 1502 out of thesheath 1508 (or proximal movement of the sheath 1508 relative to thebasket 1502) causes the basket 1502 to automatically radially expand.Correspondingly, proximal movement of the basket 1502 into the sheath1508 (or distal movement of the sheath 1508 relative to the basket 1502)causes the basket 1502 to automatically radially contract.

The ablation device 1500 includes an expandable or inflatable toroidballoon 1510. The balloon 1510 is configured to be pressurized with afluid such as saline, which expands or inflates the balloon 1510. Theballoon 1510 includes a plurality of sets of holes 1512, with each ofthe sets of holes 1512 being positioned adjacent to one of theelectrodes 1506. Each of the sets includes four holes 1512 in thisillustrated embodiment, but another number of holes is possible. Thefluid pressurizing the balloon 1510 is configured to leak out of theholes 1512. The holes 1512 are small, e.g., smaller than the electrodes1506, such that the fluid is configured to leak slowly out of the holes1512. The holes 1512 face radially outward similar to the electrodes1506 that are configured to contact and press against tissue such thatthe holes 1512 are similarly configured to abut the tissue. The fluidleaked out of the holes 1512 can thus be directed toward the tissue. Thefluid that pressurizes the balloon 1510 can be introduced into theballoon 1510 by, for example, being passed into one or more of thestrands or wires 1504 and then into the balloon 1510. In embodiments inwhich the fluid is a liquid, the balloon 1510 can include an insulatorbetween the electrodes 1506 and a fluid chamber of the balloon 1510 thatcontains the fluid therein. The insulator may help minimize an amount ofthe balloon 1510 that becomes a heat sink.

The ablation device 1500 includes first and second flexible suctiontubes 1514, 1516. The first suction tube 1514 is positioned distal tothe second suction tube 1516. Each of the suction tubes 1514, 1516includes a distal head 1514 h, 1516 h having a plurality of openingsformed therein through which suction can be provided in a proximaldirection into their respective tubes 1514, 1516. The first head 1514 hand the second head 1516 h act as weights of the first and second tubes1514, 1516, respectively, such that gravity pulls the heads 1514 h, 1516h in a same direction, which is a downward direction in the view of FIG.37 . Gravity will also pull fluid within a hollow organ or body lumen inwhich the ablation device 1500 is located such that the heads 1514 h,1516 h will be pulled in a direction in which the fluid will tend tocollect, thereby maximizing suctioning away of the fluid through thesuction tubes 1514, 1516.

As discussed herein, power provided to an ablation device's electrodeconfigured to contact and delver energy tissue can be adjustable. Inembodiments in which the ablation device is expandable/compressible,such as the ablation device 1410 of FIG. 23 , the ablation device 1490of FIG. 34 to FIG. 36 , the ablation device 1500 of FIG. 37 , and otherablation devices, the expansion/compression of the ablation device canbe correlated to the power. In this way, the expansion/compression ofthe ablation device can be controlled based on the amount of power beingprovided to the ablation device's electrode (which may include a singleelectrode or a plurality of electrodes), e.g., by adjusting a variableparameter of the algorithm for expansion/compression of the ablationdevice based on a variable parameter of the control algorithm for thepower. Correlating the expansion/compression of the ablation device tothe power may help improve control of the cautery and a depth of thecautery in tissue even in locations on the tissue where electricalconductivity is variable.

As discussed herein, the ablation device can be configured to expandautomatically when advanced distally out of a containment mechanism,such as a sheath or scope, and can be configured to compress whenretracted proximally into the containment mechanism. An amount that theablation device is advanced distally out of the containment mechanismcan thus affect an amount of the ablation device's expansion, as thecontainment mechanism will constrain the expandable portion of theablation device that is contained within the containment mechanism. Anamount of pressure an electrode on the expandable portion of theablation device applies to tissue can thus also be affected by an amountthat the ablation device is advanced distally out of the containmentmechanism since less than full expansion corresponds to less electrodepressure. Similarly, an amount that the ablation device is retractedproximally into of the containment mechanism can thus affect an amountof the ablation device's compression, as the containment mechanism willconstrain the compressible portion of the ablation device that iscontained within the containment mechanism. An amount of pressure anelectrode on the expandable portion of the ablation device applies totissue can thus also be affected by an amount that the ablation deviceis retracted proximally into the containment mechanism since lessretraction corresponds to more electrode pressure.

As discussed herein, a surgical device such as an ablation device can becontrolled by a controller of a surgical hub, a robotic surgical system,or other computer system, such as by retracting/advancing the ablationdevice according to at least one variable parameter of the ablationdevice's algorithm, such as a variable parameter corresponding to anamount of retraction/advancement (e.g., 0% advancement, 10% advancement,25% advancement, 50% advancement, 74% advancement, 100% advancement,etc.), a variable parameter corresponding to a rate of advancement,and/or a variable parameter corresponding to a rate of retraction. Theablation device's position relative to the containment mechanism can becontrolled by the controller controlling an amount that the ablationdevice is advanced distally out of the containment mechanism orretracted proximally into the containment mechanism, thereby alsocontrolling an amount of the ablation device's electrode pressure ontissue. The expansion/compression of the ablation device can becorrelated to the power provided to an ablation device's electrode bythe variable parameter for amount of advancement/retraction beingadjusted based on the current value of the variable parameter for power.For example, a fixed relationship can be preset between the variableparameter for the amount of retraction/advancement and the variableparameter for power such that in response to the variable parameter forpower increasing or decreasing, the variable parameter for the amount ofretraction/advancement correspondingly increases or decreases.

In some embodiments in which an ablation device is used in a surgicalprocedure, fluid expulsion can be controlled. In an exemplaryembodiment, the fluid is saline. Controlling fluid expulsion may helpimprove electrical coupling of tissue and the ablation device'selectrode (which can be a single electrode or a plurality ofelectrodes). Controlling fluid expulsion can include controlling a flowrate of the fluid and/or a salinity (hypotonic or hypertonic) of thefluid, e.g., by adjusting at least one variable parameter. Delivery ofthe fluid can be accomplished using two separate fluid feeds each for adifferent fluid. The two feeds can be combined in any manner desireddepending on a desired salt content to improve conductivity or saltcontent. For example, a first fluid feed can be for a high salinitysaline, and a second, separate fluid feed can be for distilled water.

Various embodiments of accomplishing fluid delivery are discussedfurther in U.S. Pat. No. 10,751,117 entitled “Electrosurgical InstrumentWith Fluid Diverter,” issued Aug. 25, 2020 and U.S. Pat. Pub. No.2019/0099209 entitled “Bipolar Electrode Saline Linked Closed LoopModulated Vacuum System” published Apr. 4, 2019, which are each herebyincorporated by reference in their entireties.

An ablation device, such as the ablation device 1410 of FIG. 23 , theablation device 1490 of FIG. 34 to FIG. 36 , the ablation device 1500 ofFIG. 37 , and other ablation devices, can include an electrodeconfiguration that is configured to improve electrode contact withtissue. The electrode can include an electrode array so as to include aplurality of electrodes, such as with the ablation device 1490 of FIG.34 to FIG. 36 , the ablation device 1500 of FIG. 37 , and other ablationdevices. Controlling electrode contact for an electrode array may helpimprove overall contact of each of the electrodes to the tissue beingablated.

For example, electrode contact control for an electrode array can beachieved using conformal changes in each electrode.

For another example, electrode contact control for an electrode arraycan be achieved by adjusting a rate of fluid (e.g., saline) flow and/oradjusting a salinity of fluid (e.g., saline) being delivered to a siteof the electrode contact, as discussed above.

For yet another example, electrode contact control for an electrodearray can be achieved by changing a pressure of an ablation device'sballoon to improve tissue contact of the ablation device's electrodes.As one example, changing the pressure of a balloon can be achieved usingholes in the balloon, such as the holes 1512 of the balloon 1510 of theablation device 1500 of FIG. 37 . The holes 1512 allow fluid in theballoon 1510 to leak out of the balloon 1510 such that the balloon'spressure will decrease over time (if no additional fluid is introducedinto the balloon 1512). The contact the electrodes 1506 have with tissuewill thus decrease over time as the balloon 1512 deflates/compressessince the electrodes 1506 are attached to the balloon 1510. Thisdecreased electrode contact may help prevent the tissue from overheatingand/or from nearby tissue not intended for ablation from becomingoverheated. As another example, changing the pressure of the balloon canbe achieved by using a segmented balloon in which the balloon includes aplurality of segments each configured to be independentlyinflated/expanded and independently deflated/compressed. Each of theballoon segments can have at least one electrode attached thereto suchthat the s balloon segment's associated electrode(s) can have theircontact controlled by the balloon segment's inflation/expansion anddeflation/compression. The independently controllable balloon segmentscan allow off-center pressures to be provided, which may accommodate anirregular interior circular shape of a hollow organ or body lumen. Theballoon segments can be arranged to form a toroid, such as by beingarranged in a flower petal radial pattern, so as to extend 360° forcomplete perimeter control.

For still another example, electrode contact control for an electrodearray can be achieved by using a vacuum to pull tissue into contact withthe ablation device's electrodes. A vacuum can be achieved, for example,using suction through at least one suction tubes, such as by using thesuction tubes 1514, 1516 of the ablation device 1500 of FIG. 37 . Thevacuum can originate from a single source, but in such instances thevacuum can be segmented such that individual vacuum channels can followindividual electrodes. Such segmentation may minimize blockages and/ormay allow suction to continue for one or more vacuum channels and theirassociated electrodes to improve electrode contact when one or moreother vacuum channels associated with other electrodes are not applyingsuction since electrode contact is already sufficient. Variousembodiments of using a vacuum are discussed further in previouslymentioned U.S. Pat. No. 10,751,117 entitled “Electrosurgical InstrumentWith Fluid Diverter,” issued Aug. 25, 2020 and U.S. Pat. Pub. No.2019/0099209 entitled “Bipolar Electrode Saline Linked Closed LoopModulated Vacuum System” published Apr. 4, 2019.

For another example, electrode contact control for an electrode arraycan be achieved using forward and distal balloon occlusion of a lumen toprovide for location suction at each electrode's ablation zone.Positioning the occlusion in a range of about 1 cm to about 3 cm beyondthe electrode's ablation zone can allow for intraluminal conformance tothe electrode array.

For still another example, electrode contact control for an electrodearray can be achieved by measuring electrode contact quality and, basedon the measurement, taking an action to improve electrode contactquality. Electrode contact quality can be measured, for example, using areturn electrode monitoring (REM) system in a return pad circuit. Acontact quality problem can be identified based on contact resistancebetween any two sets of the electrodes being determined to besignificantly different than other sets of the electrodes. In responseto identifying a contact quality problem, a responsive action can beautomatically taken, e.g., by a controller of a surgical hub, a roboticsurgical system, or other computer system controlling the ablationdevice and/or other relevant device. Examples of responsive actionsinclude changing a pressure of the ablation device's balloon, moving theablation device rotationally and/or translationally to readjustelectrode position and thereby readjust electrode contact, andintroducing saline to the problematic electrode contact area. Variousembodiments of return electrodes are discussed further below.

For yet another example, electrode contact control for an electrodearray can be achieved using an adaptive force application structureoperatively coupled to the electrode array and configured to apply anoutwardly directed force proportionate to a temperature measured locallyto the adaptive force application structure, such as tissue temperature.Tissue temperature can be measured in a variety of ways, as discussedherein.

The adaptive force application structure can be printed usingfour-dimensional (4D) printing. A 4D printed object is a 3D printedobject that can change structure over time. The material(s) with whichthe 4D printed structure is printed are configured to change whenexposed to a particular condition such as heat, magnetic energy, water,light, or other condition. For example, an ablation device's basket,such as the basket 1502 of the ablation device 1500 of FIG. 37 , can be4D printed using a material configured to change in response totemperature. The basket can thus be configured to compress or expand inresponse to particular temperatures, thereby allowing the electrode(s)attached to the basket to have their tissue contact adjusted based onwhether the basket expands to increase electrode tissue contact orcompresses to decrease electrode tissue contact. For another example, anablation device's basket, such as the basket 1502 of the ablation device1500 of FIG. 37 , can be made from a shape memory material configured tochange shape in response to temperature changes. As ablation occurs, theheat being applied can cause the shape memory material to change shape,thereby changing the contact of the ablation device's electrode(s) withtissue, such as by causing the electrode(s) to be pushed radiallyoutwardly to urge the electrode(s) into contact with the tissue.

Instead of being 4D printed with material(s) configured to respond totemperature change, the 4D printed structure can be printed withmaterial(s) configured to respond to magnetism to control electrodecontact. As discussed herein, a surgical procedure in which an ablationdevice is used in a duodenum or other anatomic structure can alsoinclude a laparoscope or other imaging device positioned outside theduodenum or other anatomic structure in which the ablation device islocated. A magnet can be introduced through a working channel of thelaparoscope or other imaging device outside the duodenum or otheranatomic structure in which the ablation device is positioned. Themagnet can then be used to move the 4D printed structure within theduodenum or other anatomic structure in which the ablation device ispositioned by repelling the magnetic structure.

Instead of being 4D printed with magnetic material(s), one or moremagnets can be attached to an ablation device, such as on each arm orwire of an electrode arrray. A magnet can be introduced through aworking channel of the laparoscope or other imaging device outside theduodenum or other anatomic structure in which the ablation device ispositioned. The magnet can then be used to move the structure to whichthe one or more magnets within the duodenum or other anatomic structurein which the ablation device is positioned by repelling the one or moremagnets.

Various embodiments of using a magnetic element located outside a holloworgan or body lumen and a magnetic element located into the hollow organor body lumen are discussed further below.

For yet another example, electrode contact control for an electrodearray can be achieved by each electrode of the electrode array beingoperatively coupled to an independent spring wire, with the spring wiresbeing connected together by a collar. The collar is configured toselectively advance distally and retract proximally so as to selectivelycause the electrode array to retract (collar advanced distally) orexpand (collar retracted proximally). The collar can thus simulate theelectrode array being advanced distally out of a sheath or scope orretracting proximally into a sheath or scope.

For yet another example, electrode contact control for an electrodearray can be achieved using a sleeve or stent positioned around an outerdiameter of a duodenum or other anatomic structure in which an ablationdevice is located to ablate tissue. The sleeve or stent so positioned isconfigured to provide a more uniform surface against which theelectrodes can press from inside the duodenum or other anatomicstructure than the tissue alone can provide, which may allow for moreprecise and consistent energy delivery by allowing for more uniformtissue contact.

The sleeve or stent can be positioned around the outer diameter in avariety of ways. For example, the sleeve or stent can be an elongatemember that a surgical device advances around the outer diameter with afree distal end of the elongate member leading the advancement. The freedistal end wraps around the outer diameter to return to and releasablyattach to the device, such as with a magnet. The free distal end canthereafter be released from the device to unwrap the elongate memberfrom around the outer diameter so the elongate member can be removedfrom the patient's body. The sleeve or stent can thus be similar to asizer that a laparoscopic sizing tool, such as the LINX® LaparoscopicSizing Tool (available from Ethicon US LLC of Cincinnati, Ohio),positions around an outer diameter of a tissue.

A location of where to position the sleeve or stent outside the duodenumor other anatomic structure at a location that corresponds to aninternal ablation location can be determined using a fiducial marker ona balloon or other element of the ablation device that is positionedinside the duodenum or other anatomic structure, for example on theballoon 1412 of the ablation device 1410 of FIG. 23 . The fiducialmarker can be magnetic, thereby allowing the fiducial marker inside theduodenum or other anatomic structure to be located magnetically fromoutside the duodenum or other anatomic structure without naked eye orvisible light visualization of the fiducial marker or the balloon towhich the fiducial marker is attached. The sleeve can be configured tocommunicate its location to a controller of a surgical hub, a roboticsurgical system, or other computer system, which the controller can useto verify the sleeve being properly positioned at a site of ablationsince the ablation device's position will be known.

After the ablation has been performed, the sleeve or stent can beremoved from the duodenum or other anatomic structure and from thepatient's body.

Regardless of how electrode contact is controlled, the energizing andturning off of the electrodes can be correlated with outward radialpressures being applied by the electrodes to tissue such that pressureand power can be controlled simultaneously. For example, each of theelectrodes can be configured to be energized only if a pressurethreshold for that electrode is met, e.g., if the electrode is measuredto be exerting a pressure on the tissue above a predetermined minimumpressure threshold, such that pressure and power can be controlledsimultaneously. An electrode may thus not needlessly attempt to bedelivering energy to tissue when the electrode's contact with the tissueis insufficient for effective ablation. The pressure can be measured,for example, using a pressure sensor.

Controlling Electrode Power

Devices, systems, and methods for multi-source imaging provided hereinmay allow for controlling electrode power. As discussed herein, asurgical procedure can include ablating tissue using a plurality ofelectrodes. For example, FIG. 23 illustrates an ablation device 1410including an electrode that can include a plurality of electrodes. Foranother example, FIG. 28 illustrates an ablation device 1440 includingan electrode that can include a plurality of electrodes. For yet anotherexample, FIG. 31 illustrates an ablation device including a plurality ofelectrodes 1460, 1462. For still another example, FIG. 34 to FIG. 36illustrate an ablation device 1490 including a plurality of electrodes.For another example, FIG. 37 illustrates an ablation device 1500including a plurality of electrodes 1506.

In some embodiments, the plurality of electrodes can be collectivelycontrolled such that each of the electrodes is at a same power level andis turned on/off at the same time. Such power control may simplifyenergy control, but it does not take into account that different ones ofthe electrodes may be delivering energy to tissue having differentcharacteristics, such as different temperature, different thickness,and/or different impedance, such that one or more of the electrodes isnot efficiently delivering energy to tissue and/or adjacent tissue notintended for ablation is being overly heated by the electrode's energydelivery. In some situations, one or more of the ablation device'selectrodes may not be in contact with tissue at all or may not be fullyin contact with tissue. For example, the ablation device can be expandedwithin a duodenum or other body lumen having an irregularly shaped innercircumference and/or an irregular inner surface such that one or more ofthe ablation device's electrodes is not contacting tissue at all withinthe duodenum or other body lumen or is only partially in contact withthe tissue's inner surface. Electrodes in no or only partial contactwith tissue may thus be powered improperly or unnecessarily for thatelectrode's tissue contact condition.

Controlling electrode power can include controlling each of a pluralityof electrodes individually. The electrodes may therefore each deliverenergy appropriate for the tissue with which the electrode is in contactand/or may have its power controlled to account for the electrode'stissue contact condition.

As discussed above, an algorithm stored on board an intelligent surgicaldevice, such as an intelligent ablation device, or stored elsewhere caninclude one or more variable parameters that affect control of thesurgical device. A controller of a surgical hub, a robotic surgicalsystem, or other computer system can be configured to adjust at leastone variable parameter of the algorithm to control electrode power. Eachof the intelligent ablation device's plurality of electrodes can beaffected by different variable parameters such that each of theelectrodes can be individually controlled. Examples of variableparameters related to controlling electrode power include electrodepower status (e.g., electrode on and delivering energy or electrode offand not delivering energy), rate of energy delivery, and power level(e.g., amount of power being delivered by the electrode).

The controller can be configured to receive a signal indicative of ameasured parameter and, based on the signal, determine whether or not toadjust at least one variable parameter of the algorithm to controlelectrode power and, if so determined, adjust the at least one variableparameter accordingly. Electrode power may thus be controlled based onthe measured parameter. The measured parameter can be associated with aparticular electrode, or a particular subset of the electrodes, therebyallowing the controller to adjust the at least one variable parameterfor the associated one(s) of the electrodes. Individual electrodes maythus be controlled based on the measured parameter. Examples of themeasured parameter include tissue characteristics such as temperature,thickness, and impedance.

As discussed herein, various surgical procedures can include use of anintelligent ablation device and an imaging device. For example, in aprocedure on a lung, an intelligent ablation device can be positionedinside a lung and an imaging device, such as a laparoscope, can bepositioned outside the lung. FIG. 28 illustrates one embodiment of asuch a lung procedure. For another example, in a procedure on anintestine, an intelligent ablation device can be positioned inside aduodenum and an imaging device, such as a laparoscope, can be positionedoutside the duodenum. FIG. 23 illustrates one embodiment of a such anintestinal procedure.

Images gathered by the imaging device can be used, e.g., by a controllerof a surgical hub, a robotic surgical system, or other computer system,to control power of electrodes of the ablation device. In an exemplaryembodiment, the images can indicate a depth of tissue ablation and thusindicate whether unintended layer(s) of tissue are, e.g., in danger ofbeing damaged by overheating. For example, as discussed herein, in a DMRprocedure, the mucosal layer of the duodenum is the intended ablationtarget of the duodenum while outer layers of the duodenum are notintended for ablation.

The imaging device can gather images indicative of a temperature oftissue being ablated with the images indicating ablation depth byvarying temperature levels in the imaged tissue, as discussed herein,for example with respect to FIG. 28 . For another example,intraoperative CT imaging can provide images indicative of a temperatureof tissue being ablated.

The controller can know a location and orientation of each of theelectrodes and thus be able to associate the gathered temperature datawith individual electrodes. For example, an imaging device can beconfigured to gather images from which location and orientation can bedetermined, as discussed herein. For another example, a Hall effectsensor or other sensor can be configured to sense from outside thetissue a clocking of the electrodes. The clocking can be used to relatedeach individual electrode to a position of the measured temperature torelate electrode location to measured temperature.

As discussed herein, the imaging device can be used to determine adistance to a “hidden” object, which in this instance could be thetissue and/or each of the ablation device's electrodes. The distance canbe used for any correction factor in reflectivity or temperatureradiation impacts.

The depth of tissue ablation can be used by the controller to determinewhether certain ones of the ablation device's electrodes should havetheir power adjusted. Thus, similar to that discussed herein withrespect to FIG. 28 , the controller can be configured to adjust at leastone variable parameter associated with a particular electrode to controlthat electrode's power. As discussed herein, a tissue can have varyingthickness and/or composition, so considering depth of ablation incontrolling individual electrodes may help account for differentthickness and/or composition around the tissue's circumference byallowing some electrodes to deliver more or less energy than otherelectrodes. For example, the controller can adjust at least one variableparameter to decrease or turn off power in response to the measuredtemperature for an outer, non-targeted tissue layer being above a firstpredetermined maximum threshold so as to indicate that the ablation isoverly heating or is in danger of starting to overly heat unintendedlayer(s) of tissue. For another example, the controller can adjust atleast one variable parameter to decrease or turn off power in responseto the measured temperature for an inner, targeted tissue layer intendedfor ablation being above a second predetermined maximum threshold so asto indicate that the ablation has heated the target tissue being treatedto a predetermined goal temperature or that the ablation is overlyheating or is in danger of starting to overly heat intended layer(s) oftissue. For yet another example, the controller can adjust at least onevariable parameter to increase or turn on power in response to themeasured temperature for an inner, targeted tissue layer intended forablation being less than a first predetermined minimum threshold so asto indicate that the ablation is not effectively heating for ablationthe targeted layer(s) of tissue.

As discussed herein, a sleeve or stent can be positioned around an outerdiameter of a duodenum or other anatomic structure in which an ablationdevice is located to ablate tissue. The sleeve or stent so positionedmay help mitigate any variations in the outer diameter.

As discussed herein, return electrode(s) can be positioned outsidetissue being ablated on another side of the tissue with otherelectrode(s). The return electrode(s) can be controlled similarly tothat discussed herein with respect to internally applied electrodes.Controlling energizing of the return electrode(s) may help achieve adesired sealing effect and/or may allow the return electrode(s) to bedirectionally moved to a particular tissue location to provide moreconcentrated energy at that location. Various embodiments of returnelectrodes are discussed further below.

In some embodiments, instead of or in addition to tissue temperaturebeing measured in some other way, each electrode of an ablation device'splurality of electrodes can be configured to measure tissue temperature.For example, each of the electrodes can include an integrated positivetemperature coefficient (PTC) sensor or other temperature sensor. Thecontroller may therefore be able to use the temperature measured by theelectrode in controlling the electrode, similar to that discussed abovefor controlling the electrode using temperature measured in another way.For example, the controller can adjust at least one variable parameterto decrease or turn off power for an electrode in response to thetemperature measured by that electrode being greater than apredetermined maximum threshold so as to indicate that the ablation hasheated the target tissue being treated to a predetermined goaltemperature or that the ablation is overly heating or is in danger ofstarting to overly heat the target tissue. For yet another example, thecontroller can adjust at least one variable parameter to increase orturn on power for an electrode in response to the temperature measuredby that electrode being less than a predetermined minimum threshold soas to indicate that the ablation is not effectively heating the targettissue.

The electrode including a PTC sensor may allow the electrode toself-control its energy delivery. The PTC sensor can be positioned inthe electrode's energy delivery path. The electrode self-controllingpower can be used instead of or in addition to a controller controllingelectrode power. In response to the measured temperature being greaterthan a predetermined maximum threshold, so as to indicate that theablation is overly heating or is in danger of starting to overly heatthe tissue being ablated, the resistance of the PTC will limit the powerto the electrode until (and if) the measured temperature falls below thepredetermined maximum threshold.

In some embodiments, movement of a scope through which an ablationdevice is introduced into a patient's body can be a function of powerlevel of the ablation device's electrodes, of measured temperature, oftissue impedance, of pressure of the electrodes on the tissue, and oftissue conductivity. In this way, the tissue can be ablated at differentlocations along the scope's path of travel with the scope moving betweenthe locations so as to effectively ablate the tissue at each of thelocations before moving to the next location.

FIG. 38 illustrates one embodiment of scope movement being a function ofpower level of the ablation device's electrodes, measured temperature,tissue impedance, pressure of the electrodes on the tissue, and tissueconductivity. FIG. 38 shows a duodenum 1520 being ablated such as in aDMR procedure, but other surgical procedures can be performed with scopemovement as described herein. In this illustrated embodiment, a scope1522, such as an endoscope, has been introduced into the duodenum 1520through the patient's esophagus 1524 and stomach 1526. An ablationdevice has been introduced into the duodenum 1520 through a workingchannel of the scope 1522 so as to extend distally from the scope 1522.As shown in FIG. 38 and FIG. 39 , the ablation device in thisillustrated embodiment includes first, second, third, and fourthelectrodes 1530 a, 1530 b, 1530 c, 1530 d each attached to an inflatableor expandable balloon 1544. An imaging device 1528, such as alaparoscope, is positioned within the patient outside the duodenum 1520.FIG. 38 illustrates that the visualization provided by the imagingdevice 1528 allows, as discussed herein, a first distance 1532 between anear wall 1534 of the duodenum 1520 and the imaging device 1528 and asecond distance 1536 between a far wall 1538 of the duodenum 1520 andthe imaging device 1528 to be determined.

In this illustrated embodiment, the scope 1522 is moved proximally,e.g., is retracted, in a continuous motion from its illustrated locationto a first location 1540 proximal to the illustrated location and fromthe first location 1540 to a second location 1542 proximal to the firstlocation 1540. The electrodes 1530 a, 1530 b, 1530 c, 1530 d can ablatetissue in the duodenum 1420 in more than these two locations 1540, 1542.

FIG. 40 shows a graph over time, from time t₀ to time t₁₀, indicatingpower (δ), tissue impedance (Z), tissue temperature (T) and electrodepressure (P) on tissue during ablation in which each of the fourelectrodes 1530 a, 1530 b, 1530 c, 1530 d are delivering energy in eachof the first and second locations 1540, 1542. A circle shape, a triangleshape, a rectangle shape, and a hexagon shape are shown on the lines forthe first, second, third, and fourth electrodes 1530 a, 1530 b, 1530 c,1530 d, respectively, in FIG. 40 only for identification purposes tohelp indicate which line corresponds to which electrode 1530 a, 1530 b,1530 c, 1530 d.

Controlling electrode power can include monitoring a rate of change oftemperature, which can be used to estimate tissue thickness where thetemperature was measured. The estimated tissue thickness can then beused in controlling electrode power, e.g., in changing at least onevariable parameter of an algorithm for at least one electrode of anablation device.

FIG. 41 and FIG. 42 illustrate one embodiment of controlling electrodepower using measured temperature. In this illustrated embodiment, firstand second electrodes 1550, 1552 are each positioned to contact aninterior surface 1554 of a body lumen 1556. The body lumen 1556 can be,for example, a duodenum being ablated in a DMR procedure using anablation device that includes the first and second electrodes 1550, 1552on an inflatable or expandable balloon (see for example FIG. 23 , FIG.37 , and FIG. 38 ) or other expandable member.

A temperature of an external surface 1558 of the body lumen 1556 ismeasured in this illustrated embodiment. The external surface 1558temperature is monitored at first and second locations 1560, 1562 aroundan exterior circumference of the body lumen 1556 that correspond to thelocations of the first and second electrodes 1550, 1552. The externalsurface 1558 temperature can be measured, for example, by using thermalimaging provided by an imaging device (not shown), such as alaparoscope, that is positioned outside of the body lumen 1556, by usinga temperature sensor (e.g., a temperature sensor on a flexible forceprobe or other surgical device advanced through a working channel of theimaging device), or by using an IR sensor (e.g., an IR sensor on aflexible force probe or other surgical device advanced through a workingchannel of the imaging device). A different number of electrodes can beused in other embodiments, with a corresponding different number ofexternal surface temperature measurements being gathered.

Measuring the external temperature at the first and second locations1560, 1562 can help ensure that the heating provided by the first andsecond electrodes 1550, 1552 does not overheat the tissue's outer layersthat are unintended targets of the ablation. As discussed above, thetherapeutic temperature range for tissue ablation is in a range fromabout 60° C. to about 100° C., tissue being at a temperature above about41° C. begins to make the tissue susceptible to or causes the tissue toincur irreversible cell damage, and tissue being at a temperature aboveabout 50° C. is when irreversible tissue damage begins to occur.

FIG. 41 illustrates a first tissue thickness 1566 of the body lumen 1556where the first electrode 1550 is located and where the first externaltemperature is being measured, and a second tissue thickness 1568 of thebody lumen 1556 where the second electrode 1552 is located and where thesecond external temperature is being measured. The first tissuethickness 1566 is greater than the second tissue thickness 1568 in thisillustrated embodiment.

FIG. 42 shows a graph plotting time versus measured first and secondexternal temperatures at the first and second locations 1560, 1562,respectively, and power level of the ablation device that includes theelectrodes 1550, 1552. A circle shape and a rectangle shape are shown onthe lines for the first and second locations 1560, 1562, respectively,in FIG. 42 only for identification purposes to help indicate which linecorresponds to temperature at which of the locations 1560, 1562associated with which of the electrodes 1550, 1552. The graphdemonstrates rate of change of the measured first and second externaltemperatures being used to indicate tissue thickness.

When power begins being provided to the electrodes 1550, 1552 for theelectrodes 1550, 1552 to deliver energy to the body lumen 1556 (verticalaxis line in the graph), the power is at its predetermined energy startlevel, which is 80 W in this illustrated embodiment. Reference A in thegraph shows a starting rate of change for the second measured externaltemperature, and Reference C in the graph shows a starting rate ofchange for the first measured external temperature. The tissue isthinner where the second external temperature is being measured, ascompared to where the first external temperature is being measured, sothe “A” rate of change is greater than the “C” rate of change. As shownin the graph, in response to the measured external temperature at one ofthe first and second locations 1560, 1562 reaching a predeterminedmaximum temperature threshold, the power level is reduced for theassociated electrode 1550, 1552. The predetermined maximum temperaturethreshold is 60° C. in this illustrated embodiment, but another valuecan be set, such as 41° C., 50° C., 70° C., or other value. Changing thepower level for an electrode can be accomplished by changing at leastone variable parameter of the algorithm being used to control ablation,as discussed herein. The first external temperature initially reachesthe predetermined maximum temperature threshold later than the secondexternal temperature due to first tissue thickness 1566 being greaterthan the second tissue thickness 1568, as reflected by the lowerstarting “C” rate of change. The power level for each of the first andsecond electrodes 1550, 1552 is repeatedly increased or decreased inresponse to the rate of change for each electrode's associated measuredtemperature and to the measured first external temperature (used incontrolling the first electrode's power) and in response to the measuredsecond external temperature (used in controlling the second electrode'spower). Reference B in the graph shows an ending rate of change for thesecond measured external temperature, and Reference D in the graph showsan ending rate of change for the first measured external temperature. Inresponse to detecting the ending rate of change for the second measuredexternal temperature, power is turned off for the second electrode 1552,and the second measured external temperature thereafter decreases asshown in the graph. In response to detecting the ending rate of changefor the first measured external temperature, power is turned off for thefirst electrode 1550, and the first measured external temperaturethereafter decreases as shown in the graph. The first externaltemperature initially reaches the ending rate of change later than thesecond external temperature due to first tissue thickness 1566 beinggreater than the second tissue thickness 1568.

In some embodiments, controlling electrode power using a monitored rateof change of temperature can limit or control application of heat to amucosal layer of tissue versus a serosal layer of the tissue. A flashintensity of energy can be delivered to ablate the mucosal layer withinterconnection between the tissue's layers acting as a transientboundary that changes a conductivity of the applied energy. The flash ofheat can be sufficient to kill mucosal cells in the mucosal layer, butby the time the heat dissipates to the serosal layer, the heat will notbe enough to damage the serosal layer. The flash intensity can be higherthan would normally be applied for ablation, but because it is deliveredin a very fast, flash fashion, the high amount of power can be usedwithout overly heating the serosal layer.

Controlling electrode power can include monitoring a temperaturegradient, which can then be used in controlling electrode power, e.g.,in changing at least one variable parameter of an algorithm for at leastone electrode of an ablation device.

FIG. 43 and FIG. 44 illustrate one embodiment of controlling electrodepower using a temperature gradient. In this illustrated embodiment,first and second electrodes 1570, 1572 are each positioned to contact aninterior surface 1574 of a body lumen 1576. The body lumen 1576 can be,for example, a duodenum being ablated in a DMR procedure using anablation device that includes the first and second electrodes 1570, 1572on an inflatable or expandable balloon (see for example FIG. 23 , FIG.37 , and FIG. 38 ) or other expandable member.

A temperature of an exterior surface 1578 of the body lumen 1576 atfirst and second locations 1580, 1582 around an exterior circumferenceof the body lumen 1576 that correspond to the locations of the first andsecond electrodes 1570, 1572 is measured in this illustrated embodimentsimilar to that discussed above regarding FIG. 31 and FIG. 41 . Atemperature of the interior surface 1574 of the body lumen 1576 ismeasured in this illustrated embodiment at the locations of the firstand second electrodes 1570, 1572 similar to that discussed aboveregarding FIG. 31 . A different number of electrodes can be used inother embodiments, with a corresponding different number of external andinternal surface temperature measurements being gathered.

FIG. 43 illustrates a first tissue thickness 1586 of the body lumen 1576where the first electrode 1570 is located and where the first externaltemperature is being measured, and a second tissue thickness 1588 of thebody lumen 1576 where the second electrode 1572 is located and where thesecond external temperature is being measured. The first tissuethickness 1586 is greater than the second tissue thickness 1588 in thisillustrated embodiment.

Similar to that discussed above, measuring internal and externaltemperatures of the body lumen 1576 allows a temperature gradient to beestablished from outside the serosal layer to the mucosal layer insidethe lumen 1576 such that a temperature of each tissue layer can beestablished. The internal and external temperatures will not be the samebefore ablation begins because the inner tissue layer, e.g., mucosallayer, acts as an insulator. The internal and external temperatures willnot be the same during ablation since the internal surface 1574 of thebody lumen 1576 is having heat applied thereto.

FIG. 44 shows a graph plotting time versus temperature and power levelof the ablation device that includes the electrodes 1570, 1572. Thefirst and second measured internal temperatures and the first and secondmeasured external temperatures are shown in the graph. A circle shape, arectangle shape, a hexagon shape, and a triangle shape are shown on thelines for the first and second measured external temperatures and thefirst and second measured internal temperatures, respectively, in FIG.44 only for identification purposes to help indicate which linecorresponds to temperature at which of the locations associated withwhich of the electrode 1570, 1572. The graph demonstrates temperaturegradient being used to indicate tissue thickness.

When power begins being provided to the electrodes 1570, 1572 for theelectrodes 1570, 1572 to deliver energy to the body lumen 1576 (verticalaxis line in the graph), the power is at its predetermined energy startlevel, which is 80 W in this illustrated embodiment. Power level remainsat 80 W for each of the electrodes 1570, 1572 until time (1), when atemperature gradient G1 associated with the second electrode 1572 isdetermined to not meet a predetermined temperature gradient thresholdfor the second electrode 1572 when the measured second externaltemperature reaches a predetermined maximum temperature threshold. Thetemperature gradient associated with the second electrode 1572 isdefined by a difference between the measured second external andinternal temperatures. The predetermined maximum temperature thresholdis 60° C. in this illustrated embodiment, but another value can be set,such as 41° C., 50° C., 70° C., or other value. The tissue is thinnerwhere the second external and internal temperatures are being measured,as compared to where the first external and internal temperatures arebeing measured, so the second external temperature reaches thepredetermined maximum temperature threshold before the first externaltemperature reaches the predetermined maximum temperature threshold. Asshown in the graph, in response to the temperature gradient G1associated with the second electrode 1572 not meeting, e.g., exceeding,the predetermined temperature gradient threshold for the secondelectrode 1572 when the measured second external temperature reaches thepredetermined maximum temperature threshold, the power level is reducedfor the second electrode 1572 at time (1). Changing the power level foran electrode can be accomplished by changing at least one variableparameter of the algorithm being used to control ablation, as discussedherein.

The first external temperature initially reaches the predeterminedmaximum temperature threshold at time (2), later than the secondexternal temperature at time (1), due to first tissue thickness 1586being greater than the second tissue thickness 1588. As shown in thegraph, in response to the temperature gradient G2 associated with thefirst electrode 1570 not meeting the predetermined temperature gradientthreshold for the first electrode 1570 when the measured first externaltemperature reaches the predetermined maximum temperature threshold, thepower level is reduced for the first electrode 1570 at time (2). Thetemperature gradient associated with the first electrode 1570 is definedby a difference between the measured first external and internaltemperatures.

The power level for each of the first and second electrodes 1570, 1572is repeatedly increased or decreased in response to the temperaturegradient associated with each electrode 1570, 1572 when the measuredexternal temperature associated therewith reaches the predeterminedmaximum temperature threshold.

At time (3) the temperature gradient G3 associated with the secondelectrode 1572 first meets, e.g., is less than, the predeterminedtemperature gradient threshold for the second electrode 1572 when themeasured second external temperature reaches the predetermined maximumtemperature threshold. In response, the power is turned off for thesecond electrode 1572 at time (3). The second measured external andinternal temperatures thereafter decrease as shown in the graph. At time(4) the temperature gradient G4 associated with the first electrode 1570first meets, e.g., is less than, the predetermined temperature gradientthreshold for the first electrode 1570 when the measured first externaltemperature reaches the predetermined maximum temperature threshold. Inresponse, the power is turned off for the first electrode 1570 at time(4). The first measured external and internal temperatures thereafterdecrease as shown in the graph. The temperature gradient associated withthe first electrode 1570 meets the predetermined temperature gradientthreshold later than the temperature gradient associated with the secondelectrode 1572 due to first tissue thickness 1586 being greater than thesecond tissue thickness 1588.

Controlling electrode power can include monitoring at least one opticalproperty (absorption, scattering, etc.) of the tissue being ablated,which can then be used in controlling electrode power, e.g., in changingat least one variable parameter of an algorithm for at least oneelectrode of an ablation device, similar to that discussed aboveregarding rate of change and temperature gradient.

Controlling electrode power can include monitoring external tissuetemperature with or without also monitoring internal tissue temperature,as discussed herein. In some embodiments, monitoring external tissuetemperature can include using one or more fiber optic sensors. The fiberoptic sensors can be advanced to the tissue through a working channel ofan imaging device, such as a laparoscope, positioned outside of thetissue being ablated from within, e.g., being ablated using an ablationdevice. The fiber optic sensors can include fiber optic pressure sensorsand/or fiber optic temperature sensors. One example of a fiber opticpressure sensor is the OPP-M200 fiber optic pressure sensor availablefrom Opsens Solutions Inc. of Québec, Canada. Examples of fiber optictemperature sensors include the OTG series of fiber optic temperaturesensors available from Opsens Solutions Inc. of Québec, Canada.

Each of the one or more fiber optic sensors can be positioned at alocation corresponding to where an electrode of the ablation device iscontacting tissue inside the tissue. A number of the fiber optic sensorscan thus equal a number of the electrodes. A location of where toposition each of the one or more fiber optic sensors outside the tissueat a location that corresponds to the one or more electrodes inside thetissue can be determined, for example, using a fiducial marker that ispositioned inside the tissue. Various embodiments of using a fiducialmarker to determine a location of an ablation device and/or a scopethrough which an ablation device has been advanced are discussed furtherbelow.

FIG. 45 and FIG. 46 illustrate one embodiment of using one or more fiberoptic temperature sensors to monitor external tissue temperature andusing the monitored external tissue temperature to control electrodepower. Four electrodes 1590 and four fiber optic temperature sensors1592 are shown in this illustrated embodiment, but another number ofelectrodes and fiber optic sensors can be used. One of the electrodes1590 and one of the fiber optic sensors 1592 is obscured in FIG. 45 .

FIG. 45 shows a duodenum 1594 being ablated such as in a DMR procedure,but other surgical procedures can be performed using fiber opticsensors. In this illustrated embodiment, a scope 1596, such as anendoscope, has been introduced into the patient's duodenum 1594 throughthe patient's esophagus 1598, esophageal sphincter 1600, and stomach1602. An ablation device has been introduced into the duodenum 1594through a working channel of the scope 1596 so as to extend distallyfrom the scope 1596. Each of the ablation device's first, second, third,and fourth electrodes 1590 is attached to an inflatable or expandableballoon 1604 of the ablation device. An imaging device (not shown), suchas a laparoscope, is positioned within the patient outside the duodenum1594. The fiber optic sensors 1592 have been advanced to the duodenum1594 through a working channel of an imaging device and positionedoutside of the duodenum 1594 with each of the fiber optic sensors 1592being positioned at an external surface of the duodenum at a locationcorresponding to one of the electrode's location within the duodenum1594.

FIG. 46 shows a graph of time, from time to t₀ time t_(n), versus power(in Watts) and tissue temperature (in ° C.) measured by the fiber opticsensors 1592. A circle shape, a triangle shape, a rectangle shape, and ahexagon shape are shown on the lines for the first, second, third, andfourth fiber optic sensors 1592 and the first, second, third, and fourthcorresponding electrodes 1590, respectively, in FIG. 46 only foridentification purposes to help indicate which line corresponds to whichelectrode 1590/fiber optic sensor 1592 pair. Ablation begins at time towith each of the electrodes 1590 starting to deliver energy. TemperatureT3 in the graph defines a predetermined maximum threshold that, whenmeasured by a particular fiber optic sensor 1592, triggers a controllerof a surgical hub, a robotic surgical system, or other computer systemto adjust the power for the corresponding electrode 1590, e.g., bychanging at least one variable parameter of an algorithm, so the tissuetemperature can decrease to help protect the duodenum's outer layersunintended for ablation from being overly heated. Power stops beingprovided to each of the electrodes 1590 at time t_(n), which correspondsto when each of the four measured external tissue temperatures hasreached temperature T₁, which is less than temperature T₃ and defines apredetermined minimum threshold. The temperatures are not the same foreach of the electrode 1590/fiber optic sensor 1592 pairs over time t₀ totime t_(n), indicating that the thickness of the tissue where eachelectrode 1590/fiber optic sensor 1592 pair is positioned is not thesame.

Fiber optic temperature sensors 1592 are used in the embodiment of FIG.45 and FIG. 46 , but as mentioned above, fiber optic pressure sensorscan be used. In such embodiments, pressure of the electrodes on tissuecan be measured and electrode power controlled accordingly. Fiber opticpressure sensors can be used in addition to or instead of fiber optictemperature sensors or other temperature sensing means.

For an ablation device including a plurality of electrodes, all of theplurality of electrodes can be simultaneously delivering power, one ormore of the plurality of electrodes can deliver energy while one or moreothers of the plurality of electrodes are not delivering energy. As alsodiscussed herein, the plurality of electrodes can be attached to anexpandable or inflatable member such as a basket or a balloon. FIG. 47to FIG. 50 illustrate various ones of a plurality of electrodes 1610 ofan ablation device 1616 (partially shown in FIG. 47 to FIG. 50 )attached to a basket 1612 of the ablation device 1616 and deliveringenergy with the basket 1612 in different expansion states. A distal tip1618 of the ablation device 1616 to which a distal end of the basket1612 is attached is also shown in FIG. 47 to FIG. 50 . Four electrodes1610 are shown in this illustrated embodiment, but another number ofelectrodes can be used. Depending on a size of a body lumen in which thebasket 1612 is positioned, the basket 1612 can have different amounts ofexpansion for the electrodes 1610 to each contact an internal surface ofthe tissue. Depending on measured parameter(s), different ones of theelectrodes 1610 can be simultaneously delivering energy and incombinations other than those illustrated in FIG. 47 to FIG. 50 .

FIG. 47 illustrates the basket 1612 in a first state of expansion andeach of the electrodes 1610 delivering energy in their respectiveablation zones 1614. The electrodes 1610 each have a same power and thushave same-sized ablation zones 1614. Adjacent ablation zones 1614overlap with one another. More than one electrode 1610 can thuscontribute to ablation of a same tissue location. FIG. 48 illustratesthe basket 1612 in a second, greater state of expansion and each of theelectrodes 1610 delivering energy in their respective ablation zones1614. The electrodes 1610 each have a same power and thus havesame-sized ablation zones 1614. Unlike with the basket 1612 in thefirst, smaller state of expansion, the ablation zones 1614 do notoverlap with the basket 1612 in the second state of expansion eventhough the electrodes 1610 have a same power in FIG. 47 and FIG. 48 .FIG. 49 illustrates the basket 1612 in the first state of expansion withtwo of the electrodes 1610′ not delivering energy (power off) and two ofthe electrodes 1610 delivering energy in their respective ablation zones1614. The electrodes 1610 that are delivering energy have a same powerin FIG. 47 , FIG. 48 , and FIG. 49 and thus have same-sized ablationzones 1614. FIG. 50 illustrates the basket 1612 in the second state ofexpansion with one of the electrodes 1610′ not delivering energy (poweroff) and three of the electrodes 1610, 1610″ delivering energy in theirrespective ablation zones 1614. One of the electrodes 1610″ deliveringenergy has more power than the other two electrodes 1610 deliveringenergy and thus has a larger ablation zone 1614″. The other twoelectrodes 1610 that are delivering energy have a same power as in FIG.47 , FIG. 48 , and FIG. 49 and thus have same-sized ablation zones 1614.

In some embodiments, controlling electrode power can include usingprevious ablation settings used with a particular patient and aparticular ablation device as the ablation device is used at differentlocations in the patient. Ablation may therefore be performed fasterand/or more efficiently. For example, a memory operably coupled to acontroller of a surgical hub, a robotic surgical system, or othercomputer system controlling the ablation device can store therein theone or more variable parameters of an algorithm the controller usesduring a surgical procedure to control the ablation device in ablatingtissue at a particular location in the patient. As discussed herein, theone or more variable parameters can change during performance of thesurgical procedure. After the ablation has stopped and the ablationdevice has been moved to a second location in the patient to ablate thetissue at the second location, the controller can use the stored one ormore variable parameters of the algorithm when beginning ablation at thesecond location since those variable parameter(s) have already beendetermined to be effective for that patient and that tissue. The one ormore variable parameter(s) may change during the ablation at the secondlocation but may be more likely to not need much or any adjusting byusing the previously used parameter settings.

Controlling electrode power can include monitoring a plurality ofparameters and using each of the monitored parameters to adjust thepower for each of one or more electrodes being used to ablate tissue.The plurality of parameters can include, for example, two or more oftissue impedance, external tissue temperature (e.g., as measured usingimaging, using a fiber optic temperature sensor, using a temperaturesensor, etc.), internal tissue temperature (e.g., as measured usingimaging, using a temperature sensor, etc.), and tissue pressure (e.g.,as measured using a fiber optic pressure sensor, using a pressuresensor, etc.).

Controlling electrode power can include communicating measurements ofeach monitored parameter to a generator supplying energy to an ablationdevice's one or more electrodes.

In some embodiments, an end effector of a surgical device can include anelongate shaft and opposing jaws that are at the distal end of theelongate shaft. Such an end effector has a dual jaw configurationbecause the end effector includes two jaws. The jaws are configured tomove between open and closed positions. One or both of the jaws can bemovable to move the jaws between the open and closed positions. The jawsinclude at least one electrode configured to deliver energy to tissueengaged between the jaws. The delivered energy seals the tissue, such assealing after the tissue has been cut by a cutting element of thesurgical device. The end effector including at least one electrode canhave a variety of configurations.

FIG. 51 illustrates one embodiment of an end effector 1700 includingopposed upper and lower jaws 1702, 1704 configured to engage tissuetherebetween. FIG. 51 shows the end effector 1700 open. The upper jaw1702 includes a positive electrode 1706 configured to contact tissueengaged between the jaws 1702, 1704, and the lower jaw 1704 includes anegative electrode 1708 configured to contact tissue engaged between thejaws 1702, 1704. Controlling electrode power for the positive electrodecan be similar to that discussed above regarding an ablation device'selectrode.

FIG. 52 illustrates another embodiment of an end effector 1710 includingopposed upper and lower jaws 1712, 1714 configured to engage tissue 1716therebetween. FIG. 52 shows the end effector 1710 closed. One of both ofthe upper and lower jaws 1712, 1714 includes a segmented electrode. Forexample, as shown in FIG. 53 , FIG. 53A, and FIG. 54 , the end effector1710 can have a multi-source, multi-return configuration in which theupper jaw 1712 includes a segmented positive electrode including foursegments 1718 and the lower jaw includes a segmented negative electrodeincluding four segments 1720. Another number of segments can be used.FIG. 53 and FIG. 54 show the end effector 1710 closed. For anotherexample, as shown in FIG. 55 and FIG. 56 , the end effector 1710 canhave a single-source, multi-return configuration in which the upper jaw1712 includes one positive electrode 1722 and the lower jaw includes asegmented negative electrode including four segments 1724. Anothernumber of segments can be used. FIG. 55 and FIG. 56 show the endeffector 1710 closed. For yet another example, as shown in FIG. 57 andFIG. 58 , the end effector 1710 can have a multi-source, single-returnconfiguration in which the upper jaw 1712 includes a segmented positiveelectrode including four segments 1726 and one negative electrode 1728.Another number of segments can be used. FIG. 57 and FIG. 58 show the endeffector 1710 closed.

Controlling electrode power for a segmented electrode can include eachof the electrode segments being controlled independently, similar tothat discussed above regarding independent control of an ablationdevice's plurality of electrodes.

Controlling electrode power for an end effector having a dual jawconfiguration can include monitoring for collateral thermal damage andusing the monitored collateral thermal damage as a control for the powerapplied to the tissue engaged between the jaws. In general, monitoringfor collateral thermal damage includes monitoring at least oneparameter, e.g., temperature, impedance, etc., at an external surface ofthe tissue and controlling power based on the monitored at least oneparameter, as discussed herein.

Controlling electrode power for an end effector having a dual jawconfiguration can include using an area of tissue engaged between thejaws to monitor one or more parameters of the tissue and using themonitored one or more parameters to control electrode power. In anexemplary embodiment, the parameter(s) are monitored outside an energyzone (similar to an ablation zone) of a particular electrode such thatone or more properties of tissue near the tissue intended to beenergized by the electrode can be used to control the electrode's power.In other words, tissue not in the return path of the electrodedelivering energy can be used in controlling the electrode's power. Thenearby tissue may thus be protected from being unintentionally damagedby the electrode's energy delivery while allowing the electrode to applyenergy effective to seal the intended tissue. For example, for an endeffector engaging tissue between its jaws and including a segmentedelectrode, tissue contacting one of the segments can be monitored tocontrol another one of the segments.

Examples of the monitored parameter include frequency response,capacitance, pressure, temperature, and impedance. Embodiments of themonitored parameter including at least one of pressure, impedance, andtemperature are discussed elsewhere herein. Embodiments of the monitoredparameter including at least one of frequency response and capacitanceare discussed further below.

As mentioned above, one example of a parameter that can be monitored intissue to control an electrode is frequency response. In someembodiments, frequency response can be used as a detecting ornon-therapeutic sweep before or between therapeutic energy applicationsto tissue engaged by the jaws. FIG. 59 illustrates one embodiment ofusing frequency response to monitor tissue between therapeutic energyapplications. A first low power measurement pulse 1730 (e.g., in a rangeof about 10 Hz to about 1000 Hz) is applied to tissue near tissueintended to be energized, such as by one electrode segment applying thepulse 1730 to tissue intended to energized by another electrode segment.A controller of a surgical hub, a robotic surgical system or othercomputer system that is controlling energy delivery can use the firstlow power measurement pulse to determine a current tissue state using,for example, one or more of implied impedance via voltage/currentsampling, signal reflection and measurement (similar to Doppler radar),infrared capacitance measurement, and multiple frequencies. After a timedelay T_delay, an energizing, higher frequency treatment pulse 1732 isdelivered to the tissue, with the pulse being based on the determinedcurrent tissue state. After another time delay T_delay following thedelivery of the energizing treatment pulse 1732, a second low powermeasurement pulse 1734 is applied to tissue near tissue intended to beenergized, with the process repeating until energy delivery ceases.

FIG. 60 illustrates another embodiment of using frequency response tomonitor tissue between therapeutic energy applications. In thisillustrated embodiment, nested multi-frequency signals are applied 1740via one or more electrodes with a discrete therapeutic frequency powerlevel so as to be piggybacked onto the discrete therapeutic frequencypower level. FIG. 61 illustrates one embodiment of multi-frequencyapplication using a multiplexor (e.g., on board the surgical device orat a generator supplying energy to the surgical device) and threefrequencies. A rate of change of the multi-frequency signals ismonitored 1742 and used in controlling electrode power, namely either bydetermining whether to apply 1744 power or not apply 1746 power. Poweris not applied 1746 if there is a short, otherwise power is applied1744. A source of the multi-frequency signals is known, so the rate ofchange can be determined, e.g., by a controller of a surgical hub, arobotic surgical system, or other computer system controlling energydelivery. A low frequency pulse (e.g., in a range of about 10 Hz toabout 1000 Hz) will have a different lower impedance than the tissue insweeping the lower frequencies in a low impedance versus shortcondition. A tissue may respond optimally to a certain frequency overanother frequency. An imaging device visualizing the tissue can be usedto filter out an appropriate activation frequency.

FIG. 62 shows schematically the process of FIG. 60 using the endeffector 1700 of FIG. 51 as an example. A first impedance sensor 1748measures impedance (local or remote) on a delivery side, and a secondimpedance sensor 1750 measures impedance (local or remote) on a returnside. A low impedance condition differentiates between an electrodeshort (do not apply 1746 power) versus low impedance (apply 1744 power).

FIG. 63 and FIG. 64 illustrate another embodiment of using frequencyresponse to monitor tissue. In this illustrated embodiment, a variablefrequency measurement pulse 1760 and a therapeutic treatment pulse 1762at a fixed frequency are applied to tissue 1764 at a same time. Acombiner or multiplexor 1766 is used to combined the variable frequencypulse and the therapeutic treatment pulse 1762. Radiofrequency (RF) isused as the energy in this illustrated example but other energy ispossible. A high pass filter 1768 differentiates between the high powertherapeutic treatment pulse 1762 and the variable frequency sensingpulse 1760. A rate of change of the variable frequency sensing pulse1760 is used in controlling electrode power similar to that discussedabove.

One embodiment of using a high frequency measurement pulse 1780 and atherapeutic treatment pulse 1782 to determine a short (do not applypower) or low impedance (apply power) is illustrated in FIG. 65 . FIG.65 shows the high frequency measurement pulse 1780 relative to thetherapeutic treatment pulse 1782 in the time domain. The high frequencymeasurement pulse 1780 includes four baseline signals in thisillustrated embodiment. Typical frequency dependence on permittivity andconductivity of tissues is discussed further in, for example, Miklavčičet al., Wiley Encyclopedia of Biomedical Engineering, “ElectricProperties of Tissue,” John Wiley & Sons, Inc., 2006, p. 1-12, which ishereby incorporated by reference in its entirety.

FIG. 66 , FIG. 67 , and FIG. 68 illustrate embodiments of a measuredacceptable condition, a measured fault condition, and a measuredmarginal condition, respectively, for the measurement pulse 1780. In themeasured acceptable condition, which indicates that power can bedelivered, the measured responses show variation in magnitude, phase,and profile across the four baseline signals across various frequencieseven with the first frequency signal being unchanged. In the measuredfault condition, which indicates that power should not be delivered, thefrequency response fails to show a variation below a threshold level1784, thereby indicating a short. In the measured marginal condition,which indicates that power can be delivered, shorting exists at certainfrequencies (frequencies 2 and 3) but not at other frequencies(frequencies 1 and 4). Power delivery may be acceptable because onlysome frequencies detect a variation while other frequencies do not, solikelihood of a short is small Some frequencies may fail while others donot due to a condition such as an RF Open presenting as a short due to aquarter wave stub.

Nested multi-frequency signals are applied in the embodiment of FIG. 65to FIG. 68 , but the condition analysis described can be similarly usedwith a detecting or non-therapeutic sweep before or between therapeuticenergy applications. For example, FIG. 69 shows in the time domain afirst measurement pulse 1786, a therapeutic treatment pulse 1788 appliedafter a first time delay, and a second measurement pulse 1790 after asecond time delay.

FIG. 70 and FIG. 71 illustrate one embodiment of providing a variablefrequency measurement pulse and a controller 1792 (e.g., of a surgicalhub, a robotic surgical system, or other computer system) receiving datain response therefrom that the controller can use in determining whetherto apply power to tissue 1794. The variable frequency measurement pulseis provided in this illustrated embodiment by an RF source 1796, whichmay be provided, for example, via an endoscope. The RF source 1796generates a first frequency (Freq=1) at a first time and loops throughadditional frequencies from a second frequency (Freq=2) at a second timethrough an Nth frequency (Freq=N) at an Nth time, where N is an integergreater than two. An RF antenna 1798 broadcasts the generated firstthrough Nth frequencies. Receiver antennas 1800 each tuned to one of thefirst through Nth frequencies and positioned outside the tissue 1794receive the signal broadcast at the frequency to which the receiverantenna 1800 is tuned. The received signals each pass through acorresponding tuned bandpass filter 1802 and through ananalog-to-digital converter 1804 before being passed to the controller1792. Since the construction and arrangement of the receiver antennas1800 is known a priori, the relative positioning of the receiverantennas 1800 to the RF antenna 1798 can be known. Based on changes insignal properties (amplitude and phase delay), the signals received bythe controller 1792 provide information regarding the tissue 1794 in thedirection indicated by the relative positioning.

As mentioned above, one example of a parameter that can be monitored intissue to control an electrode of a dual jaw end effector iscapacitance. Dielectric change can be used to determine a type of thetissue engaged by the jaws and to control electrode power. Anon-therapeutic RF signal, e.g., a signal with power below the levelthat induces therapeutic effects on the tissue, can be delivered to thetissue, e.g., by an electrode on one of the jaws, to determine a densityor a change of tissue type along the jaws. A ratio of power in theelectrode to capacitance of tissue adjacent to the electrode can be usedto balance pressure, conductivity, or power.

For example, tissue engaged by the jaws can have variablecompressibility and thickness due to adhesions or chronic disease.Measuring a rate of change of capacitance adjacent to the electrode thatwill deliver therapeutic energy can be used to determine betweenvariation of pressure or variation of power to complete the electrodeweld. Upper and lower thresholds can be used to induce differenteffects.

For another example, resistance versus parasitics (parasitic capacitanceand parasitic inductance) can be measured during energy application, asthe ratio may change during the energy application due to the tissue'svariable compressibility and thickness. Power delivery may therefore notbe as expected. A shift in frequency of the power based on the ratio mayminimize the parasitic leaching effect. The tissue could have a highimpedance at low frequency, a low impedance at high frequency, or viceversa, which enables the controller to tune the frequency to the tissueto improve the power level's effectiveness on the tissue.

Providing a variable frequency measurement pulse that can be received bya tuned antenna array, similar to that discussed above regarding FIG. 70, may allow for detection of the tissue's orientation and properties.Also, filters may be used as part of the electrodes, which may allow theRF source, e.g., a generator, to have full output with the filterscontrolling measures.

In some embodiments in which one or more parameters of tissue aremonitored, a previously sealed area of tissue can be used to monitor andcontrol sealing of an adjacent area of the tissue. A previously sealedarea of tissue has functional characteristics of a denatured zone havinghigher impedance and lower conductivity since collagen has already fusedand water has been removed, thereby allowing for a more stablemeasurement albeit a measurement that may be less sensitive. In someembodiments in which one or more parameters of tissue are monitored, anarea of tissue that has not yet been sealed or that is not intended forsealing (non-targeted tissue) can be used to monitor and control sealingof an adjacent area of the tissue. Such a tissue area not yet sealedwill have more water, a higher conductivity, and lower impedance than apreviously sealed area of tissue and will therefore be more sensitive tomonitoring effects of the adjacent area of tissue. In some embodimentsin which one or more parameters of tissue are monitored, both apreviously sealed area of tissue and an area of tissue that has not yetbeen sealed or that is not intended for sealing can be used to monitorand control sealing of an adjacent area of the tissue.

Monitoring tissue adjacent an area of tissue to be sealed can beaccomplished, for example, using a first, non-therapeutic set ofelectrodes on an edge of a jaw of the end effector, while a second,therapeutic set of electrodes on the jaw located radially inward of thefirst set of electrodes can be used to seal the intended, targetedtissue. The non-therapeutic set of electrodes can “float” on theenergized state of the therapeutic circuit. An isolation element such asa transformer can be used to power the non-therapeutic set ofelectrodes.

The therapeutic set of electrodes can have a high impedance coating toprevent therapeutic high power flow through while allowing for lowcurrent sensing. An aspect of the high impedance coating can becharacterized to determine if an individualized resistive fingerprintwould be able to respond to the higher sensing signal.

Controlling electrode power for an end effector having a dual jawconfiguration can include monitoring a power parameter or electrodeaspect of the surgical device's connection to a return path or agenerator supplying energy to the surgical device, thereby allowingdistally controlled power delivery to be monitored. Return lossmonitoring (remote monitoring away from the surgical site) may thereforebe performed for a monopolar array.

Optimizing impedance of the source with the tissue may maximizeeffective power delivery to the surgical device by allowing aninadvertent change of tissue path return to be identified. A ratio ofthe delivered power (power supplied to the surgical device) and thereflected power (power returned back from the surgical device) can matchimpedance to the patient, which may allow for maximum power efficiency.If delivery efficiency is detected to suddenly shift, the energy focalpoint is likely to have shifted. Current would show an inadvertent shortto trigger power level adjustment, while impedance would show aninadvertent change of tissue path return and not trigger power leveladjustment.

Scope and Electrode Location Monitoring and Control

Devices, systems, and methods for multi-source imaging provided hereinmay allow for scope and electrode location monitoring and control.

As discussed herein, a surgical procedure can include a scope and anablation device positioned in a hollow organ or a body lumen that isbeing visualized from an external point of view (extraluminalvisualization) using an imaging device. For example, in a DMR procedure,a scope such as an endoscope can be positioned in a duodenum, anablation device including an electrode (which may be a single electrodesor a plurality of electrodes) can be positioned in a duodenum distal tothe scope, and an imaging device such as a laparoscope can be positionedexternal to the duodenum. In other surgical procedures, the scope andthe ablation device can be positioned in a different hollow organ orbody lumen.

The scope and the ablation device within the hollow organ or body lumencan be difficult to visualize from within the hollow organ or bodylumen, e.g., due to curvature of the hollow organ or body lumen and/ordue to the limited space within the hollow organ or body lumen to allowan imaging device to be positioned within the hollow organ or body lumento achieve a full view or even a partial view of the scope and/or theablation device. Therefore, it can be difficult to determine whether theelectrode(s) of the ablation device are properly positioned before beingenergized to ablate target tissue within the hollow organ or body lumenbecause the location of the electrode(s) may not be known, and/or it canbe difficult to determine that each intended target of ablation withinthe hollow organ or body lumen has been ablated as intended because itmay not be known whether the scope has moved enough within the holloworgan or body lumen to allow the ablation device to access and ablateeach target.

The imaging device's visualization of the scope and/or the ablationdevice from outside the hollow organ or body lumen can be used todetermine a location of the scope and/or the ablation device within thehollow organ or body lumen. A location of electrode(s) of the ablationdevice can thus be determined before the electrode(s) are energized toablate target tissue within the hollow organ or body lumen and/or whilethe electrode(s) are energized and ablating target tissue, which mayhelp ensure that the electrode(s) are properly located to ablate thetarget tissue. In addition to or instead of determining the location ofthe scope and/or the ablation device, the imaging device's visualizationof the scope and/or the ablation device can be used to control movementof the scope and/or the ablation device within the hollow organ or bodylumen, which may help ensure that each intended target of ablationwithin the hollow organ or body lumen is reached for ablation.

In some embodiments, scope and electrode location monitoring and controlcan include controlling scope movement based on at least one parametermonitored from outside a hollow organ or body lumen in which the scopeis positioned. An imaging device positioned outside the hollow organ orbody lumen can be configured to gather images, as discussed herein, andthereby monitor the at least one parameter. A controller incommunication with the imaging device and the scope can receive a signalfrom the imaging device regarding the monitored parameter(s). Thecontroller can receive the signal directly from the imaging device orthrough one or more intermediary devices. As discussed above, analgorithm stored on board the scope or stored elsewhere can include oneor more variable parameters. The controller can be configured to adjustat least one variable parameter of the algorithm based on the monitoredparameter(s), as indicated by the received signal. The at least onevariable parameter can be related to movement of the scope within thehollow organ or body lumen, such as advancement rate (rate of distalmovement) or retraction rate (rate of proximal movement). Movement ofthe scope can thus be controlled based on information gathered by theimaging device despite the imaging device being located outside thehollow organ or body lumen in which the scope is positioned.Consequently, a location of an ablation device advanced through thescope and/or advanced outside the scope and positioned distal to thescope can thus also be controlled, which may help ensure that eachintended target for ablation is ablated.

The parameter monitored using the imaging device's visualization caninclude one or more of, for example, tissue temperature, current flow intissue, tissue impedance, tissue thickness, and tissue water density.For example, the imaging device can be configured to gather thermalinformation using, e.g., an infrared (IR) camera, to monitor atemperature of an external surface of the hollow organ or body lumen inwhich the scope and the ablation device are located. The images can begathered while the ablation device is delivery energy to the tissue,e.g., using one or more electrodes contacting an internal surface of thetissue, so as to be heating the tissue. In response to the externaltemperature reaching a predetermined maximum threshold, the controllercan cause the scope and/or the ablation device to move, e.g., to beretracted, and can adjust at least one variable parameter of thealgorithm to adjust a rate of the scope's and/or ablation device'smovement based on a rate of change of the monitored temperature.Monitoring the tissue's temperature using IR thermal imaging can also beused to determine a width of the energy seal provided by the ablationbased on the starting and stopping temperatures monitored.

FIG. 28 illustrates one embodiment in which thermal information gatheredby an imaging device can be used to control position of the ablationdevice. As discussed above, FIG. 28 illustrates an ablation device(ablation probe) 1440 positioned in a lung and illustrates an imagingdevice 1444 that is positioned outside the lung and that is configuredto gather images using at least infrared light, e.g., by using an IRcamera. As indicated in the graph of FIG. 28 , the IR thermal cameramonitors the temperature of an external surface of the lung as shown bythe “Lung Tissue” line in the temperature versus time portion of thegraph. The graph also shows position of the ablation device 1440 versustime. In response to the measured external surface temperature at timet₂ reaching a predetermined maximum threshold, which is 41° C. in thisillustrated embodiment, the ablation device's position is changed, suchas by a controller of a surgical hub, a robotic surgical system, orother computer system causing movement of the ablation device or of ascope in which the ablation device 1440 is located. At time t₂, theablation device 1440 is shown in the graph to move in the x, y, and zdimensions. In response to the measured external surface temperature attime t₄ again reaching the predetermined maximum threshold, the ablationdevice's position is again changed. At time t₄, the ablation device 1440is shown in the graph to move in the y and z dimensions.

In some embodiments, scope and electrode location monitoring and controlcan include controlling a centering of an ablation device's electrodeswithin a hollow organ or body lumen.

As discussed herein, an ablation device can include a plurality ofelectrodes. For example, the ablation device 1410 of FIG. 23 can includea plurality of electrodes attached to the balloon 1412. For anotherexample, the ablation device 1490 of FIG. 34 includes a plurality ofelectrodes. For yet another example, the ablation device 1500 of FIG. 37includes a plurality of electrodes 1506.

For still another example, as shown in FIG. 72 , an ablation device caninclude a balloon 1810 and a plurality of electrodes 1812. FIG. 72 showsa longitudinal axis 1810A of the balloon 1810, which is coaxial with alongitudinal of the ablation device. The electrodes 1812 are segmentedin this illustrated embodiment so can be independently controlled, suchas by providing power to only certain ones of the electrodes 1812 viapower lines extending distally for operative coupling with a powersupply. The balloon 1810 in this illustrated embodiment is formed of aflexible circuit material. The electrodes 1812 in this illustratedembodiment are spaced equidistantly around a circumference of theballoon 1810 and are printed on an outer surface of the flexible circuitmaterial.

The ablation device in this illustrated embodiment also includes aplurality of fiducial markers 1814 that are printed on the outer surfaceof the flexible circuit material. The fiducial markers 1814 can beotherwise applied to the balloon's outer surface, such as being a smallcoil adhered to the outer surface of the balloon 1810 (in which case thematerial can but need not be flexible circuit material). The balloon1810 is configured to selectively expand and compress by selectivelyintroducing fluid into and withdrawing fluid from an interior of theballoon 1810. The balloon 1810 is enclosed except at a valve 1816 thatcan be selectively opened to allow fluid introduction and withdrawal. Insome embodiments, the fluid can be hot water, which when inside theballoon 1810 can heat the electrodes 1812 enough for the electrodes 1812to ablate tissue without being supplied with energy from a power supply.

Centering an ablation device's electrodes within a hollow organ or bodylumen may help maximize contact of each of the electrodes against aninterior surface of the hollow organ or body lumen, thereby helping toensure that ablation occurs around an entire inner circumference of thehollow organ or body lumen. A longitudinal axis of the ablation device'sballoon or other expandable member can be used in centering the ablationdevice's electrodes within the hollow organ or body lumen since theelectrodes are attached to the balloon or other expandable member.Coaxially aligning the longitudinal axis of the ablation device'sballoon or other expandable member with a longitudinal axis of thehollow organ or body lumen in which the ablation device is positionedwill center the ablation device's electrodes within a hollow organ orbody lumen. The hollow organ or body lumen's longitudinal axis can beknown through imaging, such as via visualization provided by the imagingdevice positioned outside the hollow organ or body lumen, and/or by acentered projection line visualized by the imaging device. Theprojection line can be projected distally, for example, by a scopethrough which the ablation device has been advanced and from which theablation device distally extends. A controller of a surgical hub, arobotic surgical system, or other computer system in communication withthe imaging device can thus know each of the hollow organ or bodylumen's longitudinal axis and the longitudinal axis of the ablationdevice's balloon or other expandable member, thereby allowing thecontroller to move the ablation device so the longitudinal axes arecoaxially aligned and thus so the electrodes are centered.

The ablation device's electrodes can be centered in a variety of ways.For example, each of the electrodes can be configured to emit a lowlevel electromagnetic pulse. The imaging device located outside thehollow organ or body lumen in which the ablation device is located canreceive the emitted pulses, such as with an electromagnetic sensor, toallow each of the electrode's positions to be determined, such as by acontroller of a surgical hub, a robotic surgical system, or othercomputer system in communication with the imaging device, since astrength of the magnetic field indicates relative distances of eachelectrode to the receiver. Based on the electrodes' positions, thecontroller can cause the ablation device to move within the hollow organor body lumen to center the electrodes.

For another example, each of an ablation device's fiducial markers canbe magnetic and used to detect location of the ablation device. Thefiducial markers can, for example, be attached to the ablation device'sballoon or other expandable member. The imaging device located outsidethe hollow organ or body lumen in which the ablation device is locatedcan include a magnetoresistive sensor configured to determine locationof the fiducial markers, and thus location of the balloon or otherexpandable member and the electrodes thereon, based on the magneticsignatures of the fiducial markers. Based on the balloon or otherexpandable member's position, the controller can cause the ablationdevice to move within the hollow organ or body lumen to center theelectrodes.

A location of the fiducial markers can facilitate determination of theballoon or other expandable member's location and thus facilitatedetermining location of the electrodes. For example, in the embodimentof FIG. 72 , a first fiducial marker 1814 (in an upper left position inthe view of FIG. 72 ) is positioned at a rear or proximal end of theballoon 1810, a second fiducial marker 1814 (in a bottom right positonin the view of FIG. 72 ) is positioned at a front or distal end of theballoon 1810, a third fiducial marker 1814 (in a center positon in theview of FIG. 72 ) is positioned equidistantly between the front and rearends of the balloon 1810, and a fourth fiducial marker 1814 ispositioned at a front or distal end of the electrodes 1812. The fourthfiducial marker 1814 positioned relative to the electrodes 1812 forfacilitating determination of electrode location has a smaller size thanthe first, second, and third fiducial markers 1814 positioned relativeto the balloon 1810 for facilitating determination of balloon 1810location. The fiducial markers 1814 can be detected as discussed herein,thereby allowing the controller to determine a location of the balloon1814, e.g., based on the larger first, second, and third fiducialmarkers 1814, and a location of the electrodes 1812, e.g., based on thecentered, third and the smaller, fourth fiducial markers 1814. In someembodiments, the fourth fiducial marker 1814 and/or the third fiducialmarker 1814 can be omitted, while in other embodiments, the first,second, and third fiducial markers 1814 can be omitted.

In some embodiments, scope and electrode location monitoring and controlcan include detecting completion of ablation to determine when to stopsupplying power to the ablation device's electrode(s) so as to stopablation. For example, CT imaging provided by an imaging devicepositioned outside the hollow organ or body lumen in which the ablationdevice is positioned can gather thermal images indicative of tissuetemperature. The CT imaging device can be located entirely outside thepatient, such as with intraoperative CT imaging using a C-arm. Inresponse to the measured temperature reaching a predetermined maximumtemperature indicative of ablation completion, power can stop beingsupplied to the ablation device's electrode(s).

In some embodiments, scope and electrode location monitoring and controlcan include using a magnet. The magnet can allow for determiningmovement and/or location, and/or can for determining tissue thickness.

FIG. 73 illustrates one embodiment of location monitoring and controlusing a magnet. FIG. 73 shows an ablation device 1820 positioned in aduodenum 1822 of a patient, but a magnet can be similarly used in otherhollow organs and body lumens. The ablation device 1820 includes anexpandable member 1824, in the form of a basket, to which a plurality ofelectrodes are attached. A first magnet 1826 is at a distal tip of theablation device 1820 distal to the expandable member 1824. South (S) andnorth (N) poles of the first magnet 1824 are shown in FIG. 73 .

An ablation device can be advanced into a hollow organ or body lumenthrough an overtube and/or a scope (e.g., a working channel of thescope). In this illustrated embodiment, the ablation device 1820 isadvanced into the duodenum 1822 through an overtube 1828. An endoscope1830 has also been advanced through the overtube 1828 and is alsopositioned in the duodenum 1822. However, the endoscope 1830 cannotvisualize the expandable member 1824 (or any of the electrodes attachedthereto) as positioned in FIG. 73 due to the curvature of the duodenum1822 and the relative positions of the endoscope 1830 and the expandablemember 1824.

As shown in FIG. 73 , a surgical device 1832 is positioned outside ofthe duodenum 1822. The surgical device 1832 can be so positioned in anyof a variety of ways, such as by being advancing laparoscopicallythrough a laparoscope 1834, as in this illustrated embodiment. Thesurgical device 1832 includes a second magnet 1836 at a distal tipthereof. South (S) and north (N) poles of the second magnet 1836 areshown in FIG. 73 . The second magnet 1836 is configured to be movedoutside the duodenum 1822 to cause movement of the first magnet 1826,and thus the expandable member 1824 and the electrodes, within theduodenum 1822 by magnetically interacting with the first magnet 1826.The second magnet's movement can be any combination of rotation (shownby a first arrow 1838), translational movement (shown by a second arrow1840), or lateral movement (shown by a third arrow 1842).

The movement of the first magnet 1826 in response to the movement of thesecond magnet 1836 depends on a relative position of the north and southpoles of the first and second magnets 1826, 1836. FIG. 74 illustratesthe first magnet 1826 in the duodenum 1822 in a passive configuration inwhich the second magnet 1836 is not magnetically interacting with thefirst magnet 1826. FIG. 75 illustrates the first magnet 1826 in theduodenum 1822 in an attraction configuration in which the second magnet1836 is positioned relative to the first magnet 1826 such that the firstmagnet 1826 is attracted to the second magnet 1836. Therefore, the firstmagnet 1826, and thus the expandable member 1824 and the electrodesattached thereto, have moved closer to an interior wall of the duodenum1822 in a direction toward the second magnet 1836. In FIG. 75 the firstmagnet 1826 is attracted to the second magnet 1836 by the south (S) poleof the second magnet 1836 being positioned adjacent to the north (N)pole of the first magnet 1824 with a tissue wall of the duodenum 1822being positioned therebetween. Instead of the south (S) pole of thesecond magnet 1836 being positioned adjacent to the north (N) pole ofthe first magnet 1824 in the attraction configuration, the north (N)pole of the second magnet 1836 can be positioned adjacent to the south(S) pole of the first magnet 1824. FIG. 76 illustrates the first magnet1826 in the duodenum 1822 in a repulsion configuration in which thesecond magnet 1836 is positioned relative to the first magnet 1826 suchthat the first magnet 1826 is repulsed by the second magnet 1836.Therefore, the first magnet 1826, and thus the expandable member 1824and the electrodes, have moved closer to the interior wall of theduodenum 1822 in a direction away from the second magnet 1836. FIG. 76also shows with a fourth arrow 1844 (similar to the first arrow 1838)that the second magnet 1836 has been rotated from its position in FIG.75 so as to cause the first magnet's movement from the attractionconfiguration to the repulsion configuration. In FIG. 76 the firstmagnet 1826 is repulsed by the second magnet 1836 by the north (N) poleof the second magnet 1836 being positioned adjacent to the north (N)pole of the first magnet 1824 with a tissue wall of the duodenum 1822being positioned therebetween. Instead of the north (N) pole of thesecond magnet 1836 being positioned adjacent to the north (N) pole ofthe first magnet 1824 in the repulsion configuration, the south (S) poleof the second magnet 1836 can be positioned adjacent to the south (S)pole of the first magnet 1824.

In some embodiments, a magnetic element can be attached to an ablationdevice configured to be positioned within a hollow organ or body lumen,such as in the embodiment of FIG. 73 . In other embodiments, a magneticelement can be attached to a scope configured to be positioned within ahollow organ or body lumen. The magnetic element being attached to thescope may allow a location of the scope to be tracked within the holloworgan or body lumen from outside the body lumen and/or may allow tissuethickness to be determined.

FIG. 77 illustrates one embodiment of a scope 1850 that includes a firstmagnet 1852. The first magnet 1852 in this illustrated embodiment is inthe form of a magnetic collar extending circumferentially around thescope 1850 just proximal to a distal end of the scope 1850. FIG. 77 andFIG. 78 show the scope 1850 positioned within a hollow organ or bodylumen 1854 and with an expandable member 1856, in the form of a basket,of an ablation device extending distally from the scope 1850. The firstmagnet 1852 is configured to be magnetically detected from outside thehollow organ or body lumen 1854 using a second magnet 1858. The secondmagnet 1858 in this illustrated embodiment includes a plurality ofmagnets in a chain configured to be wrapped circumferentially around anexternal surface of the hollow organ or body lumen 1854, as shown inFIG. 78 . The second magnet 1858 can be wrapped around the externalsurface in a variety of ways, such as similar to that discussed aboveregarding a sleeve or stent being positioned around a hollow organ orbody lumen's outer diameter.

The second magnet 1858 is configured to move along the hollow organ orbody lumen's external surface corresponding to movement of the scope1850, and thus of the first magnet 1852, within the hollow organ or bodylumen 1854 due to the attraction of the first and second magnets 1852,1858. FIG. 79 illustrates one embodiment of scope 1850 movement withinthe hollow organ or body lumen 1854. The scope 1850 is retractedproximally from a first, distal position to a second, proximal positionas shown by an arrow 1860. The first and second magnets 1852, 1858 thusalso move proximally The first, distal position of the scope 1850′, thefirst magnet 1852′, and the second magnet 1858′ is noted by thoseelements being numbered with an apostrophe.

A first magnet positioned within a hollow organ or body lumen and asecond magnet positioned outside the hollow organ or body lumen with atissue wall located between the first and second magnets are configuredto cooperate to allow determination of a thickness of the tissue wall. Astrength of magnetic attraction between the first and second magnetswill vary based on a thickness of the tissue wall therebetween. Thus,strength of the magnetic attraction at different locations along anaxial length of the hollow organ or body lumen can indicate a thicknessof the tissue wall at that location.

For example, the first and second magnets 1852, 1858 of FIG. 77 to FIG.79 are configured to allow determining thickness of the hollow organ orbody lumen 1854 positioned between the first and second magnets 1852,1858. As shown in FIG. 77 , a thickness of the hollow organ or bodylumen 1854 is different at different axial locations along a length ofthe hollow organ or body lumen 1854. The tissue has a first thickness1862 at a first axial location (1), a second thickness 1864 at a secondaxial location (2) proximal to the first axial location (1), and a thirdthickness 1866 at a third axial location (3) proximal to the secondaxial location (2). In this illustrated embodiment the first thickness1862 is less than the third thickness 1866, which is less than thesecond thickness 1868. As the scope 1850 is retracted (moved proximally)within the hollow organ or body lumen 1854, as discussed above, themagnetic attraction between the first and second magnets 1852, 1858varies as indicated in a graph shown in FIG. 80 plotting tissuethickness and power (magnetic attraction) for each of the first, second,and third positions (1), (2), (3). Different magnetic attractionsbetween the first and second magnets 1852, 1858 are known for each of aplurality of different tissue thicknesses, so detected magneticattractions can be correlated to known tissue thicknesses, such as byusing a lookup table stored in a memory accessible to a controller of asurgical hub, a robotic surgical system, or other computer system.

In addition to or instead of using a magnet for scope and electrodelocation monitoring and control, ultrasound imaging can be used todetermine scope and/or electrode location. The ultrasound imaging can beused to locate a scope and/or electrode(s) within a hollow organ or bodylumen. The magnet can then be positioned outside the hollow organ orbody lumen near the determined location and used to control electrodemovement, as discussed above. Additionally or alternatively, theultrasound imaging can be used to locate electrode(s) within a holloworgan or body lumen during the magnetically controlled electrodemovement to confirm the electrode movement visually, e.g., by display ofgathered ultrasound images. FIG. 81 illustrates one embodiment of anultrasound imaging device 1870 visualizing through a tissue wall 1872,such as an abdominal wall for visualizing a duodenum or other portion ofan intestine.

In some embodiments, location monitoring and control can includecontrolling ablation device rotation. Controlling rotation of anablation device may help control cauterization exposure and/or may helpensure complete ablation of an internal surface of a hollow organ orbody lumen around a circumference thereof. When a portion of theinternal surface is determined to be ablated, such as by temperaturemonitoring, the ablation device can be rotated so an electrode that wasablating the now-completed area of tissue can now deliver energy toanother area along the internal surface of the tissue. A rate and/oramount of the rotation can be controlled by adjusting at least onevariable parameter of a control algorithm. The ablation device'srotation can be controlled via rotation of the ablation device, viarotation of a scope in which the ablation device is positioned (such asin a working channel thereof) so as to rotate the ablation device withthe scope, or via rotation of an overtube in which the ablation deviceis positioned (such as in an inner lumen thereof) so as to rotate theablation device with the overtube. An ablation device can also betranslated longitudinally, as discussed herein, to help ensure that alltarget tissue is ablated.

Devices and systems disclosed herein can be designed to be disposed ofafter a single use, or they can be designed to be used multiple times.In either case, however, the devices can be reconditioned for reuseafter at least one use. Reconditioning can include any combination ofthe steps of disassembly of the devices, followed by cleaning orreplacement of particular pieces, and subsequent reassembly. Inparticular, the devices can be disassembled, and any number of theparticular pieces or parts of the device can be selectively replaced orremoved in any combination. Upon cleaning and/or replacement ofparticular parts, the devices can be reassembled for subsequent useeither at a reconditioning facility, or by a surgical team immediatelyprior to a surgical procedure. Those skilled in the art will appreciatethat reconditioning of a device can utilize a variety of techniques fordisassembly, cleaning/replacement, and reassembly. Use of suchtechniques, and the resulting reconditioned device, are all within thescope of the present application.

It can be preferred that devices disclosed herein be sterilized beforeuse. This can be done by any number of ways known to those skilled inthe art including beta or gamma radiation, ethylene oxide, steam, and aliquid bath (e.g., cold soak). An exemplary embodiment of sterilizing adevice including internal circuitry is described in more detail in U.S.Pat. No. 8,114,345 issued Feb. 14, 2012 and entitled “System And MethodOf Sterilizing An Implantable Medical Device.” It is preferred thatdevice, if implanted, is hermetically sealed. This can be done by anynumber of ways known to those skilled in the art.

The present disclosure has been described above by way of example onlywithin the context of the overall disclosure provided herein. It will beappreciated that modifications within the spirit and scope of the claimsmay be made without departing from the overall scope of the presentdisclosure. All publications and references cited herein are expresslyincorporated herein by reference in their entirety for all purposes.

What is claimed is:
 1. A surgical system, comprising: a surgical deviceincluding first and second jaws configured to engage a target tissuetherebetween, the target tissue being at a surgical site, and thesurgical device including a first electrode array configured to deliverradiofrequency (RF) energy to the target tissue; a second electrodearray configured to monitor, during the energy delivery, a non-targetedtissue at the surgical site; and a controller configured to control theenergy delivery of the first electrode array based on the monitoring ofthe second tissue by the second electrode array.
 2. The system of claim1, wherein the second electrode array includes a filter or a gatingelement configured to prevent the energy delivered by the firstelectrode array from infiltrating the second electrode array.
 3. Thesystem of claim 1, wherein a return path of the first electrode array isseparate from a return path of the second electrode array.
 4. The systemof claim 1, wherein the second electrode array includes a temperaturesensor configured to monitor a temperature of the non-targeted tissue;and the controller is configured to adjust the energy delivery inresponse to the monitored temperature being greater than a predeterminedthreshold temperature.
 5. The system of claim 1, wherein the controlincludes the controller controlling power level and frequency of theenergy delivery.
 6. The system of claim 1, wherein the control includesthe controller controlling frequency of the energy delivery.
 7. Thesystem of claim 1, wherein the second electrode array is configured tomonitor impedance of the non-targeted tissue; and the controller isconfigured to adjust the energy delivery in response to the monitoredimpedance as compared to a threshold impedance.
 8. The system of claim1, wherein the second electrode array is configured to monitor afrequency response of the non-targeted tissue; and the controller isconfigured to adjust the energy delivery based on the frequencyresponse.
 9. The system of claim 1, wherein the second electrode arrayis configured to monitor at least one of capacitance and pressure of thenon-targeted tissue; and the controller is configured to adjust theenergy delivery based on the monitored at least one of capacitance andpressure.
 10. The system of claim 1, wherein the controller isconfigured to cause the control by adjusting a variable parameter of acontrol algorithm of the surgical device; and the control algorithm isconfigured to, when executed, affect the energy delivery from the firstelectrode array to the tissue.
 11. The system of claim 1, wherein asurgical hub includes the controller.
 12. The system of claim 1, whereina robotic surgical system includes the controller, and the surgicaldevice is configured to releasably couple to and be controlled by therobotic surgical system.
 13. A surgical method, comprising: deliveringradiofrequency (RF) energy to tissue at a surgical site with a firstelectrode array of a surgical device engaging the tissue between jaws ofthe surgical device; monitoring a parameter of non-targeted tissue atthe surgical site using a second electrode array; and adjusting, with acontroller, the energy delivery to the tissue based on the monitoredparameter.
 14. The method of claim 13, wherein the second electrodearray includes a filter or a gating element that prevents the energydelivered by the first electrode array from infiltrating the secondelectrode array.
 15. The method of claim 13, wherein a return path ofthe first electrode array is separate from a return path of the secondelectrode array.
 16. The method of claim 13, wherein the secondelectrode array monitors at least one of impedance, frequency response,capacitance, temperature, and pressure of the non-targeted tissue; andthe controller is configured to adjust the energy delivery based on themonitored at least one of impedance, frequency response, capacitance,temperature, and pressure.
 17. The method of claim 13, wherein theadjusting includes adjusting a variable parameter of a control algorithmof the surgical device; and the method further comprises executing thecontrol algorithm including the adjusted variable parameter, therebyaffecting the energy delivery from the first electrode array to thetissue.
 18. The method of claim 13, wherein a surgical hub includes thecontroller.
 19. The method of claim 13, wherein a robotic surgicalsystem includes the controller, and the surgical device is releasablycoupled to and controlled by the robotic surgical system.